When to Use AI Context Compression — and When Caching Beats It

Managing large language model context windows efficiently requires choosing between several techniques, including deletion, retrieval, caching, and compression. Lossless cleanup — stripping markup, minifying structured data, and deduplicating chunks — can reduce token counts by 30–60% before any model-based method is applied. Model-based approaches such as extractive selection, abstractive summarisation, and token pruning (notably Microsoft Research's LLMLingua) offer higher compression ratios but carry costs and quality trade-offs. A cost comparison shows that caching a stable 20,000-token context is both cheaper and lossless compared to compressing it, making compression most justified when context changes on every request or must be shared across models. The recommended decision order is: delete useless tokens first, retrieve instead of stuffing context, cache what is stable, and only then compress what remains.