New Attention Technique Cuts Transformer KV Cache Memory by 45% With Minimal Accuracy Loss

Grouped Value Attention (GVA) is a new approach to transformer memory management that reduces persistent key-value cache scalars by approximately 45–47% compared to existing grouped-query attention methods. Instead of storing a full key matrix, GVA stores only grouped values and reconstructs keys on the fly during decoding. Tested on a 350-million-parameter model across five tasks, GVA matched GQA's average accuracy to within 0.01 points. Separately, DeepSeek-V4.1-Flash achieves even greater compression, shrinking its KV cache footprint to around one-eighth of its predecessor while reportedly improving performance. Both approaches show promise for enabling longer context windows on less powerful hardware, though real-world latency and throughput gains on commodity GPUs have yet to be confirmed.