Developer builds AI context selector, benchmark shows it beats grep on larger datasets

A developer built 'cognitive-cache', a token-budget-aware file selector for coding tools that scores repo files against a task using six weighted signals without any LLM calls or API keys. The tool was benchmarked against grep and TF-IDF retrieval strategies using recall and MRR metrics on curated GitHub issues. Initial tests on 23 hand-curated issues showed no statistically significant advantage over simple keyword grep, prompting a switch to the larger SWE-bench Lite dataset. On a 78-instance SWE-bench subset, cognitive-cache achieved a recall@5 of 0.538 versus grep's 0.359, with a confidence interval excluding zero, confirming a real improvement. The project also revealed a flaw in its own budget selector, which was discarding relevant files its own ranking had already surfaced, highlighting the gap between ranking quality and end-to-end retrieval performance.