Dev team found AI rules file pointed to deleted script for 24 days, undetected by shape tests

A software development team discovered that one of their AI agent skill files referenced a deleted command for 24 days, during which 28 commit-time shape tests passed without flagging the issue across 142 commits. The problem came to light after the team audited 707 script references across their CLAUDE.md, rules files, and skills, finding that shape tests verify file structure but cannot detect whether named commands still exist in the codebase. To address this gap, the team built a 26-line bash script called check-script-refs.sh that cross-checks all pnpm, npm run, and yarn references in Markdown documentation against the actual scripts defined in package.json. The script identifies missing or stale command references by file and line number, exits with an error code when mismatches are found, and can be integrated into pre-commit hooks or CI pipelines. The audit and tooling were run on September 22, 2026, using Claude Code 2.1.278, and currently report zero missing script references in the team's repository.
This is an AI-generated summary. ShortSingh links to the original source for the complete article.
Discussion (0)
Log in to join the discussion and vote.
Log in