Why Agent Timeouts Should Be Treated as Correctness Failures, Not Just SLO Metrics

A DEV Community article argues that latency is widely overlooked as a failure mode in AI agent evaluation, even though timing out is one of the most common ways production agents fail. The author contends that missing a deadline is a correctness failure, not merely an infrastructure concern, since a late response can be worse than no response at all. Drawing on a tiered evaluation framework, the piece places deadline compliance in the highest-trust tier alongside checks like valid JSON output, because wall-clock time is a signal the agent itself cannot manipulate. Most evaluation harnesses miss this entirely because they only receive the final output, by which point timing data has already been discarded. The article proposes wrapping agent calls in a structured result type that captures elapsed time and timeout status, so evals grade the full run rather than just the return value.