Prewarming a Voice Agent Cut Model Costs but Not First-Reply Latency

A voice agent team found that prewarming the LLM before calls connected accounted for nearly 60% of their model spend, even though many calls ended before any conversation began. Most calls failed at voicemail, no answer, or within a turn or two, and retries made up about a third of volume with a high failure rate. They disabled prewarm for retries, and latency for prewarmed and non-prewarmed calls turned out nearly identical, with p50 around 1.2 seconds in both cases. A controlled test reproduced the same result, and prompt caching was already hitting about 98% on prewarm replies, so the prompt was warm regardless. The finding suggested the expected speed benefit was not materializing, prompting a closer look at where the delay actually came from.