Products, laws and numbers here change fast. Checked October 2026.
In plain words
Apps built on AI often send the same long text again and again: a page of instructions, a product manual, or the whole chat so far, with only a small new question tacked on the end. Normally the model has to process every word of it, every single time.
With prompt caching, the provider remembers its work on that repeated opening part for a few minutes. When the next request starts with exactly the same text, it skips straight to the new bit. The answer arrives sooner and the repeated part is billed at a discount.
You don't see this as a chat user. It matters to developers using an AI API, where long, repetitive prompts are common and costs add up.
Everyday example
A tutor who has already read the textbook
A tutor helping you through a chapter reads it once. Each time you ask a new question, they don't reread the whole chapter; they only think about your question. If you switch to a different chapter, they have to read again. Prompt caching is the tutor remembering the chapter, as long as you stay on the same one.
Where the example breaks: A tutor remembers the gist even if you skip a page. A cache only helps when the opening text is exactly identical; change one word near the start and everything after it is processed fresh.
How it actually works
What is cached is not the answer. It is the model's internal working on the prompt's opening, so each reply is still freshly generated.
Exact match only
Anthropic's docs: "Cache hits require 100 percent identical prompt segments". OpenAI's: "Cache reuse requires the entire rendered prefix to match." One changed character early on ends the reuse from that point.
Stable first, changing last
So developers order prompts carefully: tools and system prompt first, then fixed documents, then the conversation, with the new question at the very end. OpenAI advises: "Put stable developer instructions and shared reference material first."
Short memory
Caches expire. OpenAI's 2024 launch said caches are "typically cleared after 5-10 minutes of inactivity"; Anthropic's default lifetime is 5 minutes, refreshed each time it is used, with a longer paid option. Newer OpenAI models keep caches longer by default.
- Arrived
- Google Gemini context caching, May 2024. Anthropic prompt caching, August 2024. OpenAI automatic caching, October 2024
- Savings claimed
- Anthropic's launch: costs cut "by up to 90% and latency by up to 85% for long prompts". On a 100,000-token book, first reply time fell from 11.5 to 2.4 seconds
- Discounts, as of October 2026
- Cached input is billed at a fraction of the normal input price: OpenAI says up to 95% off, and Anthropic charges 10% or less of the base price. Anthropic also charges extra to write to the cache
- Also called
- Context caching (Google), cached tokens, cached input
Questions people ask
What is the difference between prompt caching and a KV cache?
A KV cache is the model's short-term working memory while it writes one reply. Prompt caching keeps that working for a shared prompt opening across separate requests, so a later request can pick it up instead of starting over.
Does prompt caching store my answers?
No. It stores the model's internal working on the repeated part of the prompt, not finished replies. Each answer is still generated fresh, so you can get different wording each time, just faster and cheaper.
Is prompt caching automatic?
It depends on the provider. OpenAI applies it automatically on supported models. Anthropic offers an automatic option or manual markers in the request. Google offers both implicit and explicit caching. Check each provider's docs for minimum lengths.
Why did my request not hit the cache?
Usually because something early in the prompt changed: a timestamp, a user name, a reordered tool list. The cache may also have expired, or the prompt was shorter than the provider's minimum length. Move changing parts to the end.