How an Unstable System Prompt Was Silently Killing LLM Cache Efficiency

A developer discovered that cached_tokens was returning near-zero values despite warm conversations, meaning full prompt costs were being charged on every turn for a roughly 16,000-token prefix. Initial suspicion fell on missing session-affinity headers in Fireworks' serverless API, but direct testing showed caching worked fine as long as the prefix was byte-identical. The real culprits were a workspace bootstrap block that disappeared after turn one and query-dependent memory being prepended directly to the system message, both of which broke prefix stability. The fix involved freezing the static system prompt and relocating volatile context into a late system turn injected just before each user message, which pushed cache hit rates from near zero up to 93–97%. The investigation also revealed that Fireworks' ~50% cache discount, unlike Anthropic's ~90%, makes re-sending large blocks to cache them costlier than simply omitting them.