Study finds up to half of a repo's tokens are wasted on lockfiles and test data

A developer built ctxray, a CLI that estimates how many tokens a coding agent would ingest from a repository and flags files that add cost without useful context. Scanning six popular open-source projects in November 2026, the tool found that 38 to 55 percent of tokens in several large repos came from files like lockfiles, test fixtures and generated assets, while the small express repo came back essentially clean. The author notes these totals are an upper bound, since agents read files selectively rather than the whole tree at once. The tool can generate an ignore file built from its findings, using .gitignore syntax compatible with several AI coding tools, along with a starter AGENTS.md. Its token counts use a rule-based approximation calibrated against a real tokenizer, with a reported median error of about 6 percent per file. The scanner and initialization features are free, and the author plans a paid tier for reports and CI integration.