Diffusion TTS Too Slow for Real-Time Chat, Repurposed as Voice Design Tool

A developer testing diffusion TTS for interactive avatars found it 2.5 times slower than a pre-trained Style-Bert-VITS2-based model when synthesizing a 7.5-second sentence on the same GPU. Even reducing diffusion steps from 40 to 16 left a 1.5x speed gap, and a fixed overhead of roughly 1.1 seconds persisted regardless of GPU resources, making real-time conversational use impractical. Allocating four times the compute still could not beat the pre-trained model's 0.8-second generation time, ruling out diffusion TTS for live dialogue. However, the engine's unique ability to generate voices from text captions alone — without any speaker audio — made it valuable for a design phase, where it creates voice prototypes used to build training corpora. The final pipeline uses diffusion TTS once per character during design and a lightweight pre-trained model at runtime, with deterministic caption-and-seed generation ensuring voices can always be exactly reproduced.