How to Build a Failure-Mining Loop to Keep AI Agent Eval Sets from Going Stale

AI agent evaluation sets often become outdated over time as production environments evolve, new tools are added, and user behavior shifts, while CI pipelines continue running the same static test cases. A structured failure-mining loop is proposed to channel real production failures back into the test suite through a controlled pipeline of candidate capture, human review, sanitization, and promotion. Engineers are advised to create bounded intermediate records of failure candidates rather than copying raw production conversations, and to select cases across diverse dimensions such as intent, tool failure role, and outcome severity. Deduplication via trajectory signatures helps prevent a single large incident from dominating the eval suite, while a promotion checklist ensures each case is reproducible, stripped of sensitive data, and assigned an owner. The approach also recommends separating deterministic structural checks from semantic quality graders so teams can distinguish hard failures from probabilistic regressions in CI results.