Untested Eval Rules Create False Security in AI Agent Pipelines

Developers building AI agents often write evaluation rules to catch production failures, but ship them without verifying the rules themselves work correctly. An untested eval can silently pass every run due to a bug, giving a false green signal while real failures go undetected. The article argues evals should be ranked by independence of signal: deterministic checks like valid output format and file existence form a reliable gate, while model-as-judge methods are too slow and non-deterministic for real-time use. Deterministic evals must be tested against a labeled corpus of known-good and known-bad traces to confirm they fire correctly. The core takeaway is that a green dashboard only proves the agent is healthy if the eval itself has first been proven capable of going red.