Prompt Cache Hit 86% But Made AI Pipeline Slower and 39% Pricier

A development team running multiple headless AI agents with OpenAI's Codex CLI tested whether switching to a persistent app-server with warm threads would speed up their pipeline through prompt caching. Initial results looked promising in isolation, but four rounds of controlled benchmarking revealed that the resident server made the pipeline slower than the original baseline and 39% more expensive in raw tokens. The root cause was that each pipeline stage used a different JSON schema, which invalidated the cached prefix on every turn, wiping out any caching benefit. Fixing this by unifying the schema restored cache hits to 66%, but wall time returned to baseline because accumulated thread history caused the model to process the same context twice. Removing redundant prompt embeddings to compensate reduced input tokens by only 4.7%, since the thread history already contained that information.