Voice Agents Need a Conversation-Level Latency SLO, Not Component Speed Claims

A fast model or detector does not guarantee a fast phone conversation, because the caller experiences the full chain of call setup, speech detection, turn-end judgment, tool calls, generation, and audio delivery. The article argues that teams should measure several distinct clocks, since reporting only one of them tends to optimise the wrong part of the system. Silence is ambiguous, so aggressive endpoint detection can interrupt callers while conservative detection creates dead air. Filler speech can mask delays but must not hide unbounded waits or imply that an action has completed. The author proposes defining a conversation SLO covering listening, turn-end timing, first intelligible audio, interruption handling, and recovery when dependencies slow or fail.