How WebSockets and two-pass processing power real-time speech-to-text

GenAI engineer Pranjul Rathour has detailed the architecture behind a live speech-to-text feature built for an OCR and Speech Workspace application. The system uses WebSockets to send small audio chunks from the browser to a server, which buffers them and returns partial transcripts in near real-time using a streaming model. A second, slower refinement pass runs after the user stops speaking, replacing the live draft with a more accurate, formatted final transcript. Key design choices include silence detection for segment boundaries, keeping model connections warm to target under one second for the first partial result, and showing partial text in a lighter style until segments are confirmed. Rathour emphasises that user-facing details — such as microphone permission prompts, a live audio level meter, and seamless reconnection handling — are as critical to the feature's success as the underlying model.