Why AI coding agents slow down in long sessions and five prompt-caching mistakes
Coding agents such as Claude Code and Codex, or open models run through Ollama, often become slower and costlier as a session grows, usually because the prompt cache is being discarded rather than because of the model itself. Each turn resends the full conversation, so reading input dominates the cost, and the cache only helps when the new request begins with exactly the same tokens as before. The cache matches only up to the first differing token, which means even a small change near the top, such as a timestamp in the system prompt, invalidates everything that follows. A local test on a MacBook Pro with Gemma 3 12B showed the cache cutting the tokens read per turn from 10,275 to 2,513 when only new content was appended, while a changed timestamp left almost no savings. The author notes that cloud models are faster but suffer the same penalty when caching breaks.
This is an AI-generated summary. ShortSingh links to the original source for the complete article.
Discussion (0)
Log in to join the discussion and vote.
Log in