Building a Browser-Only Pipeline That Turns Whisper Transcripts into Captioned MP4s

The author is developing Karaoke Captions, a React and TypeScript editor that transcribes and renders video entirely on the local machine. The article focuses on the implementation boundaries that carry word-by-word captions from compressed media to a finished MP4, including audio decoding to 16 kHz mono Float32 PCM and the timing conversions between seconds, milliseconds, and frames. Whisper runs through Transformers.js inside a Web Worker, which reports progress, device selection, and errors as distinct states. The worker also falls back to WebAssembly and retries in a fresh worker if WebGPU initialisation fails. Long audio is segmented around pauses and each segment's offset is added back to its word timestamps so later words are not misplaced.