Running NVIDIA's Nemotron 3.5 Transcribe Locally on a Single GPU

A DEV Community guide explains how to run NVIDIA's open-weight Nemotron 3.5 Transcribe model on one GPU to transcribe hour-long audio files in 40 languages. The model, released October 28, 2026 under the commercially usable OpenMDW-1.1 license, currently tops the Hugging Face Open ASR leaderboard. The article argues local deployment avoids per-minute API fees, which it says can total about $1,200 for 10,000 hours, and keeps sensitive recordings within a user's own network. It covers hardware compatibility across NVIDIA GPUs, the BF16, FP8, and NVFP4 checkpoints, and three runtimes: vLLM, SGLang-Omni, and Transformers. The guide also notes that Apple Silicon and 24 GB cards like the RTX 4090 are unsupported.
This is an AI-generated summary. ShortSingh links to the original source for the complete article.
Discussion (0)
Log in to join the discussion and vote.
Log in