SShortSingh.
Back to feed

Testing Claude Code skill argument substitution across 12 runs reveals key behaviors

0
·4 views

A developer ran 12 controlled tests on Claude Code version 2.1.278 on September 22, 2026, to observe exactly how argument placeholders behave inside skill bodies. Results showed that $ARGUMENTS always preserved the raw input string including quotes, while positional variables like $0 and $1 used shell-style splitting with quotes stripped. Missing positional arguments were left as literal text rather than replaced with empty values, and the argument-hint setting had no effect on what the skill body actually received. One notable finding was that bare environment-style tokens like $HOME, when passed as arguments, disappeared from positional slots but remained visible in $ARGUMENTS. The author built dedicated echo-skills to capture substituted content directly from session transcripts, noting that in one of 12 runs the model's reply did not match the actual substituted text — highlighting the value of reading raw transcripts over trusting model output.

Read the full story at DEV Community

This is an AI-generated summary. ShortSingh links to the original source for the complete article.

Discussion (0)

Log in to join the discussion and vote.

Log in

Related stories

0
ProgrammingDEV Community ·

Spring Boot Microservices: Sync Calls, Events, and AI Integration Explained

A technical guide explores how microservices built with Spring Boot can communicate effectively as applications grow in complexity. The article covers key tools including Spring Cloud OpenFeign, Service Discovery via Netflix Eureka, and Spring Cloud LoadBalancer for managing inter-service calls. It highlights why hardcoding service addresses is problematic and how logical service names resolve infrastructure coupling issues. The guide also addresses asynchronous processing, webhooks, and event-driven patterns to avoid blocking HTTP connections during long-running tasks. A real-world SaaS use case is presented, where an AI layer is isolated from the main backend using these architectural approaches.

0
ProgrammingDEV Community ·

Developer Shares C# Unit Testing Approach for School Sports Registration System

A developer has shared a set of C# unit tests built for a school sports registration system using xUnit and Moq frameworks. The tests cover model validation, ensuring that missing fields like Season and Sport are correctly flagged as invalid. Service-layer tests verify that the fee calculation logic returns accurate amounts for different season and sport combinations. Controller tests use a mock service to confirm that invalid form submissions re-render the view without calling the service, while valid submissions trigger the service and redirect to a success page. The example serves as a practical introduction to layered unit testing in ASP.NET Core MVC applications.

0
ProgrammingDEV Community ·

AeroCI Tool Lets Developers Test GitHub Actions Workflows Locally Before Pushing

A developer has built AeroCI, an open-source command-line tool designed to act as a local twin for GitHub Actions workflows, reducing the repetitive push-fail-debug cycle. AeroCI executes real shell commands and injects standard GITHUB_* environment variables, rather than simulating a successful run, and explicitly flags steps it cannot reproduce locally. The tool provides commands to initialize configuration, validate workflow structure, and run jobs in isolated environments — each job and matrix combination gets its own copy of the project directory to better mirror real CI runner isolation. An analysis mode can detect dead or duplicated steps, unused outputs, and generate a workflow complexity score. Secrets are kept out of job sandboxes by default and exposed only through a dedicated secrets context.

0
ProgrammingDEV Community ·

Why AI-Run Throwaway Tests May Beat Traditional Self-Healing Automation

A software testing perspective argues that self-healing test scripts create more problems than they solve, as automated locator fixes can silently assert unintended outcomes, producing costly false positives. The proposed alternative involves writing test cases in plain language and letting an AI agent execute them directly in a browser each time, eliminating the need for maintained scripts. Tools like the Playwright MCP server allow an AI agent to navigate a live page, follow test steps, capture screenshots, and generate a pass/fail report without any pre-written automation code. Because the agent works from the current state of the page, UI changes do not break anything — the agent simply adapts, much like a human tester would. The approach is positioned not as a full replacement for scripted tests, but as a practical method for short-lived or infrequent checks, with stable, high-value scenarios still promoted to deterministic scripts over time.