That AI cache discount may save less than you think

Prompt caching reuses work already done on part of a request. A common application sends the same instructions, documents or conversation history repeatedly, followed by a changing question. OpenAI’s original API caching announcement describes matching the longest previously processed prefix and reporting the reused amount in a cached-token usage field. The model still has to process new material and generate its response.

The order of information consequently matters. Changing text at the beginning can prevent later shared material from belonging to the same matching prefix. A useful design keeps genuinely stable context together, rather than adding a new timestamp before it on every call. The announcement’s historical prices and supported-model list should not be treated as today’s rate card. Its durable lesson is the accounting distinction: reused input, fresh input and generated output are different quantities. A large cache discount on one category cannot be applied automatically to all three.

Anthropic’s current documentation makes another part of that accounting explicit. It describes five-minute and one-hour cache lifetimes, with different write prices. A five-minute write costs 1.25 times the base input rate; a one-hour write costs twice that rate. Reads have their own model-specific prices, and using an entry can refresh its lifetime without another write charge.

Whether the longer lifetime helps depends on the interval between requests and how much stable material is reused. An entry that expires before the next request produces no read saving on that next call. Conversely, a frequently reused prefix can amortise its creation cost across many reads. Inspect actual usage records rather than estimating savings from the headline percentage. A useful cost report separates cache creation, cache reads, fresh input and output, then compares the complete task before and after caching. That is the level at which an application becomes cheaper, rather than simply displaying a favourable rate for one token type.