Prompt Caching Can Cost More If Cache Writes Never Get Read Back
A developer discovered that enabling prompt caching for an AI agent loop actually increased costs by about 25% instead of reducing them. The root cause was a volatile timestamp injected early in the system prompt, which changed the byte prefix on every request and prevented any cache hits. Prompt caching works as a prefix match over the fully rendered request, meaning any dynamic content placed before a cache breakpoint invalidates the entire cache entry. Cache writes cost 1.25x the base input price, while reads cost only 0.1x, so entries that are written but never read become a net surcharge. Developers can diagnose the issue by checking the usage fields: if cache_read_input_tokens stays at zero across repeated requests, something upstream is altering the prompt prefix.
This is an AI-generated summary. ShortSingh links to the original source for the complete article.

Discussion (0)
Log in to join the discussion and vote.
Log in