Developer builds local AI video dubbing pipeline using WhisperX and VoxCPM2
A developer has created an open-source, browser-based video dubbing pipeline called Video Dubbing Translator that runs media processing and speech synthesis locally on the user's machine. The pipeline separates vocal and background audio tracks, transcribes speech using WhisperX for word-level timing, translates the text via Google Translate, and clones a reference voice using VoxCPM2 to synthesize dubbed audio. The project recently migrated from XTTS-v2 to VoxCPM2, citing its Apache-2.0 license, native 48 kHz audio output, and reference-voice cloning capabilities as key advantages. An end-to-end test on a 15-second English clip successfully produced a Turkish-dubbed H.264/AAC video approximately 14.88 seconds long, with all four translated sentences verified by a separate ASR check. The developer notes that Google Translate remains a deliberate network-dependent step, meaning the pipeline is local-first but not fully offline.
This is an AI-generated summary. ShortSingh links to the original source for the complete article.

Discussion (0)
Log in to join the discussion and vote.
Log in