Mingxin FX100 Hits 90% Line-Rate on 100GbE, Easing LLM Inference Bottlenecks

Mingxin's FX100 all-flash NVMe-oF storage array achieved roughly 90% line-rate utilization on a single 100GbE port in benchmark tests, delivering approximately 11.25 GB/s of effective bandwidth. This figure is 60–80% higher than the sequential read speed of a typical single PCIe Gen4 NVMe drive, effectively removing the network as a primary bottleneck in LLM inference workloads. In long-context deployments involving a 480-billion-parameter model running on eight GPUs, FX100 cut median time-to-first-token by 26–32% compared to baseline storage configurations. When paired with LMCache's parallel read-patching in a single-card, 16-concurrency cold-read test using Qwen2.5-32B, TTFT dropped from 37.97 seconds to 9.30 seconds — a 4.1x improvement — while bandwidth rose 5.3x to 5.23 GB/s. The results suggest that high port utilization allows concurrent GPU requests to avoid bandwidth contention, enabling inference frameworks such as vLLM to access remote KV Cache with latency approaching that of local storage.