vLLM point releases quietly changed KV cache defaults and admission behavior for GLM-5 models
Two vLLM minor releases shipped in October and November 2026 changed behavior that affects latency and memory trade-offs without requiring users to opt in. Version 0.32.0 made GLM-5.x models default to an FP8 KV cache on SM100 hardware, halving KV memory per rank at a reported 7 to 11 percent time-to-first-token cost. Version 0.33.0 extended the same default to GLM-5.3-Flash, which the release notes linked to 9.8 percent higher end-to-end throughput. The same releases also altered admission control by no longer reserving max_tokens from the context window, and removed features such as cascade attention and Medusa. The author argues that such changes can invalidate capacity planning and recommends pinning exact versions, reading minor-release changelogs, and re-benchmarking on one's own traffic.
This is an AI-generated summary. ShortSingh links to the original source for the complete article.

Discussion (0)
Log in to join the discussion and vote.
Log in