How to Deploy Qwen3 235B MoE Model Using vLLM Across Verified GPU Configurations

Qwen3.8-2.4T-A95B is a 2.4-trillion-parameter Mixture-of-Experts model with approximately 95 billion parameters active per token, supporting a native context window of up to 262,144 tokens. Self-hosting the model requires significant distributed GPU infrastructure, as even low-precision checkpoints run into terabytes of storage. Documented deployment configurations range from 8 NVIDIA B300 or GB300 GPUs using an NVFP4 checkpoint (~1.32 TiB) to 16-GPU setups using FP8 (~2.27 TiB), with AMD MI355X also supported. The recommended serving stack uses a recent vLLM nightly build alongside Transformers 4.5 or later, with tensor parallelism and FP8 KV-cache enabled. The model's built-in Multi-Token Prediction head, when configured with three speculative tokens in vLLM, can meaningfully improve inference latency in low-latency serving scenarios.