SGLang Matches DeepInfer on DeepSeek's Own Benchmarks, With Small Throughput Gains

The SGLang team measured its inference engine against DeepSeek's DeepInfer on the same hardware, using DeepInfer's benchmark workload while keeping the methodological differences disclosed. The results show up to 1.04x prefill throughput on 4x GB300 and 4x B300 systems, and decode steps 1.1% to 7.4% shorter on 64x GB300, with other configurations within 1%. SGLang now integrates DeepSeek's open-sourced kernels, including FlashMLA, DeepGEMM, and DeepEP, along with several speculative decoding and memory-handling mechanisms. DeepInfer's documentation lists a KV cache system, prefill/decode disaggregation, and load balancing as needed for a complete inference service, and SGLang supports all three. Analysts note that raw throughput is only part of the picture, with serving teams still needing to decide where KV cache lives and how gateways route requests across replicas.