FP8 quantization boosts Qwen3-8B throughput 1.5x with no factual accuracy loss

Engineers tested whether switching Qwen3-8B inference from BF16 to FP8 quantization degraded output quality on an RTX PRO 6000 Blackwell GPU using vLLM, achieving a throughput jump from 1,725 to 2,597 tokens per second at concurrency 32. A structured 20-prompt benchmark spanning reasoning, math, code, and summarization tasks was run under greedy decoding to isolate any differences caused purely by numeric precision. Of the 20 outputs compared, 7 were byte-identical, 9 showed only minor wording or formatting variation, and 3 had small stylistic regressions such as a repeated word or a slightly off-topic list item. Critically, zero prompts produced a factual or numerical error in FP8 that BF16 had answered correctly, clearing the team's defined acceptance threshold. The authors caution that this validation applies only to the tested prompt profile and workload, and that different domains, longer contexts, or sampled decoding would each require a separate review pass.