Query Budgets for Fresh Support Retrieval: Limits That Protect Changed Evidence
The article argues that a hard query budget around each retrieval request should direct spending toward fresh, diverse chunks rather than forcing a model to process everything. It describes a funnel approach of fetching candidates, removing near-duplicates, reranking, and fitting results into a token-based context budget, using example values such as 40 candidates, 12 survivors, and 6 evidence chunks. The author stresses that these numbers are adjustable policy choices rather than universal truths, and recommends tracking candidate counts, trimmed evidence, and chunk age as operational signals. Chunking by meaning and change boundaries, storing hashes and timestamps, and separating support and freshness query paths are presented as key design practices. The piece also warns of failure modes such as context dilution and stale dominance, and advises surfacing empty retrievals as visible failures rather than silently retrying without limits.
