Developer Cuts Voice Agent Latency 5x by Streaming Sentences and Trimming Model Overhead

A developer building a local voice assistant found that fast LLM responses alone do not guarantee a smooth user experience, as delays accumulate across speech recognition, API calls, TTS synthesis, and audio playback. To address this, the developer modified the system so the backend streams text chunks to the device as the model generates them, rather than waiting for a complete response. The client splits incoming text into sentences using a regex boundary detector and immediately begins synthesizing each sentence, while a background worker overlaps audio playback with synthesis of the next sentence. Additional gains came from disabling extended model reasoning on conversational turns and only attaching the Google Search tool when the query actually required external information. The combined changes produced roughly a 5x reduction in time-to-first-byte on common chat turns and a noticeably more responsive perceived experience for the user.