Anthropic's Jacobian Lens Repurposed to Improve LLM KV Cache Memory Management

Anthropic published a paper on July 6 introducing J-space and the Jacobian lens, tools originally designed for interpretability and alignment auditing of language models. A developer independently applied these concepts to a different problem: determining which entries in an LLM's KV cache can be safely discarded during long inference sessions. Unlike existing methods such as H2O and SnapKV, which rely on past attention history, the proposed approach uses workspace content signals computed at prefill time to predict which cached tokens will matter in future reasoning. The method was validated against three pre-registered kill gates and successfully replicated across three Qwen models — Qwen2.5-7B, Qwen3-8B, and Qwen3-4B. The implementation has been integrated into EVOKE, a KV cache memory manager built on a forked version of llama.cpp.