Two caching strategies to keep RAG systems accurate without rebuilding context every call

Production RAG systems face a trade-off between caching answers for cost efficiency and risking stale responses when source documents change. A time-to-live (TTL) cache is a common workaround, but it fails because document freshness is triggered by events, not time intervals. The root problem is that standard answer caches discard provenance — the link between a cached response and the source documents it was derived from. A more reliable approach tracks which sources each cached unit cited, enabling surgical invalidation only when a relevant source actually changes. An open-source library called Coalent implements this provenance-based invalidation alongside semantic similarity matching, though the underlying concepts can be applied independently of any specific tool.