Voicetest Adds Live Audio Calls to Catch Voice Agent Errors Text Tests Miss

A voice agent can produce correct text yet still fail on a real phone call, because the speech pipeline can garble numbers, drug names, or appointment times. Text-only simulations stop at the model's intended output and never catch these errors. Voicetest now runs test cases as live WebRTC calls, where a simulated caller speaks through text-to-speech and listens through speech-to-text. Each turn records both the intended text and what was actually transcribed from the audio, and a word-level diff highlights mismatches. Results are judged against both versions, so a metric that passes on the written text but fails on the heard text reveals a speech-pipeline defect.
This is an AI-generated summary. ShortSingh links to the original source for the complete article.
Discussion (0)
Log in to join the discussion and vote.
Log in