Prompt Caching's Write Surcharge Can Outweigh Its Read Discount Without Enough Reuse
Providers advertise prompt caching as a 90% discount, but cached reads are only the second charge on the bill. Before cheap reads apply, the first request must pay a cache write fee, priced at 1.25 times base input for short-lived caches and roughly two to two-and-a-half times for long-lived ones. If a cached prompt is never reused or is reused only after expiry, the total cost can exceed what the uncached requests would have cost. Using a 100,000-token coding prompt as an example, a short cache breaks even after one read and a one-hour or twelve-hour cache after two reads. The analysis concludes that caching is a bet on reuse within the cache lifetime.
This is an AI-generated summary. ShortSingh links to the original source for the complete article.

Discussion (0)
Log in to join the discussion and vote.
Log in