Why cached tokens are so cheap
AI APIs charge as little as a tenth, sometimes a fortieth, of the normal input price for tokens they have seen before. The discount comes from skipping work the GPU already did. Most providers cache automatically, and keeping what changes at the end of the prompt gets you more hits. A cache hit only means your own organization sent the same prefix, although a 2025 audit found seven providers where caches were shared across users.
Every LLM API price sheet now has a line for cached input, and it sits far below the normal input price. On Claude Opus 5.5, input costs $4 per million tokens and a cache read costs $0.20. On Claude Fable 5.1 it is $10 against $0.25.
People sometimes read these lines as "cache input" and "cache output", but nothing on the output side is cached. The model still generates every output token, and Anthropic's documentation says the response is identical to what you would get without caching. What gets cached is the work of reading your prompt.
Do you have to turn it on?
Usually not. OpenAI caches by default on supported models. Gemini 2.5 and newer models have implicit caching turned on by default. DeepSeek caches automatically and needs no code changes.
Anthropic's API is the exception. It caches only when the request includes cache_control, either as one field at the top of the request, which caches up to the last block automatically, or on specific blocks. Claude Code sets this for you: it caches by default, and you have to set DISABLE_PROMPT_CACHING to turn it off. Gemini also has explicit caches, which you create yourself and pay to store by the hour.
There is no separate step for saving to the cache either. The first request that sends a given prefix computes it and leaves the result in the cache. That first request is the "cache write" on the price sheet. Anthropic bills it at 1.25× the input price for a 5-minute entry or 2× for a 1-hour entry, and OpenAI bills 1.25× on GPT-5.6 and later. Earlier OpenAI models and DeepSeek charge nothing extra for writes.
What the cache stores
A transformer handles a request in two phases. In prefill, it reads the whole prompt at once and computes, for every token in every layer, the key and value vectors that later tokens attend to. In decode, it writes the answer one token at a time on top of those vectors. Prefill is where a long prompt costs time and money: a 100,000-token prompt means 100,000 tokens of that computation before the first word of the answer.
Those key and value vectors, the KV cache, depend only on the tokens before them. If the next request starts with exactly the same tokens, the provider can reuse last time's vectors and skip prefill for that part. That is why every provider caches a prefix, counted from the first token, and why one changed byte near the top, such as a timestamp in the system prompt, turns everything after it into a miss.
The gain shows up in latency as well. When DeepSeek launched disk caching in 2024, it reported that the first token of a 128K-token prompt arrived in about 500 ms on a cache hit, down from 13 seconds.
Why the discount is so large
Skipping prefill removes most of the compute a long prompt needs, and providers pass most of that saving on. What they can't skip is storage. KV vectors are large, and the fastest place to keep them is GPU memory, the scarcest resource in the data center. That cost shows up in three places.
- Short lifetimes. Anthropic keeps an entry for 5 minutes by default and resets the timer for free on every read. OpenAI's in-memory retention on older models lasted 5 to 10 minutes of inactivity.
- Write premiums. The 1.25× and 2× write prices above pay for that storage. A prompt sent only once costs more with caching on than without it.
- Cheaper storage. OpenAI's 24-hour retention moves the tensors to GPU-local storage when memory runs short. DeepSeek keeps its cache on a distributed disk array and credits its MLA architecture, which shrinks the KV cache, with making that affordable. Google bills explicit caches by the hour: $4.50 per million tokens per hour on Gemini 3.1 Pro Preview.
Where the cache lives also matters. OpenAI keeps entries on individual machines and routes each request partly by a hash of its first tokens, so requests with the same prefix tend to reach the machine that has it. On models before GPT-5.6, traffic above about 15 requests per minute for one prefix can overflow to other machines, which lowers the hit rate.
The numbers
| Provider | Cache read | Cache write | Lifetime | Shared with |
|---|---|---|---|---|
| Anthropic | 0.1× input (0.05× on Opus 5.5, 0.025× on Fable 5.1) | 1.25× (5 min) or 2× (1 hour) | 5 min or 1 hour, reset on every read | your workspace (your organization on Bedrock and Google Cloud) |
| OpenAI, GPT-5.6 and later | 0.1× input | 1.25× | at least 30 min after last use | your organization |
| OpenAI, earlier models | discounted by model, "up to 95%" | no charge | 5 to 10 min in memory, or up to 24 hours extended | your organization |
| Google, Gemini 3.1 Pro Preview | $0.20 against $2.00 input | explicit caches pay storage, $4.50 per 1M tokens per hour | implicit caching is automatic | not stated |
| DeepSeek, at the 2024 launch | $0.014 against $0.14 | no charge, storage is free | hours to days | each user |
Short prompts aren't cached at all. The minimum is 512 tokens on the newest Claude models (4,096 on Opus 4.6 and Haiku 4.5), 1,024 on OpenAI's GPT-5.6 and later, and 2,048 to 4,096 on Gemini. DeepSeek stores the cache in 64-token units.
For a worked example on Claude Opus 5.5, say you ask 10 questions about a 100,000-token document within five minutes. Without caching, that is 10 × 100,000 tokens × $4 per million, or $4.00. With caching, the first question writes the document to the cache at 1.25×, which costs $0.50. The other nine read it for $0.18 in total. That makes $0.68, about 83% less. With a 5-minute entry, two requests already break even.
Getting more cache hits
A hit needs the prompt to match exactly from the first token, so the order of your prompt changes what you pay. The OpenAI, Anthropic and Google docs all give the same advice: put what stays the same at the start and what changes at the end.
- Put what is the same on every request first: tool definitions, the system prompt, reference documents and examples.
- Move what changes, such as the current time, the user's name or per-request context, to the end or into the last user message. A timestamp on the first line of the system prompt makes everything after it new on every request.
- Keep the order fixed. The same content in a different order is a different prompt, so tool lists and JSON keys shouldn't shuffle between requests.
- Anthropic caches tools, then the system prompt, then messages. Editing one tool definition invalidates the cached system prompt and messages after it. Toggling settings such as web search, or changing thinking settings, invalidates part of the cache too.
- If you place
cache_controlyourself on Anthropic, put it on the last block that doesn't change. Put it on a block that holds a timestamp and you pay for a write on every request and never get a read. Anthropic's docs list this as a common mistake. Automatic mode caches up to the last block, so it has the same problem when that block changes. Mark the position yourself in that case.
Did someone else ask the same thing?
No. A cache hit means your own organization, or on Anthropic your own workspace, sent the same prefix recently. Anthropic, OpenAI and DeepSeek all document that caches aren't shared across organizations or users, and Anthropic adds that this holds "even if they use identical prompts." Most hits come from your own conversation. Every chat turn resends the whole history, so each request starts with the one before it.
That guarantee hasn't always held. In 2025, Stanford researchers (Gu et al., ICML 2025) measured the time to first token across commercial APIs. They sent one prompt as a "victim", then a prompt sharing its prefix as an "attacker", and checked whether the attacker's answer came back faster than an uncached baseline. Seven providers shared caches globally across users, OpenAI among them through an embedding model, which was fixed after disclosure. A shared cache becomes a timing side channel: anyone who can guess part of your prompt can test the guess by how fast the response arrives. The same timing data showed that OpenAI's embedding model is a decoder-only transformer, which hadn't been public.
Per-organization isolation has its own catch for anyone building on these APIs. Every user of an app that calls the API with one key shares that key's cache, and studies in 2026 looked at exactly this in API gateways and relays that pool many customers behind shared credentials. If tenants in your app shouldn't learn anything about each other's prompts, separate them where the provider separates caches, for example one Anthropic workspace per tenant, or keep tenant data out of cached prefixes.
Other details worth knowing
- Cached tokens still count toward OpenAI's tokens-per-minute rate limit.
- Anthropic measures an entry's lifetime from the start of the request, so a 4-minute generation leaves about one minute for the next request to hit a 5-minute entry.
- Because a read resets the timer, traffic that arrives at least every five minutes keeps a 5-minute entry alive indefinitely, and the 1-hour option only pays off for longer gaps.
- The minimum length doesn't shrink steadily with newer models. A 3,000-token prompt caches on Claude Opus 5 and silently doesn't on Opus 4.6.
- To check that caching works, read the usage fields:
cache_read_input_tokenson Anthropic,cached_tokenson OpenAI,prompt_cache_hit_tokenson DeepSeek. A zero after repeated identical requests usually means something near the top of the prompt changes every time.
Sources: Anthropic prompt caching, OpenAI prompt caching, Gemini context caching, Gemini pricing, DeepSeek context caching on disk, Claude Code environment variables, Gu et al., Auditing Prompt Caching in Language Model APIs.
