Prompt-Caching Economics
Prompt caching lets the provider reuse computation over a repeated prompt prefix, cutting both cost and latency on cache hits by roughly an order of magnitude. It quietly reshapes how you should structure prompts and order an agent's context.
When a request reuses a prefix the model has seen recently, the provider serves the cached key-value state instead of recomputing it. The economics are lopsided: on Anthropic, a cache read costs about 0.1x the base input price, while a cache write costs 1.25x (5-minute) or 2x (1-hour). So the first call pays a small premium to write, and every subsequent hit pays roughly a tenth. OpenAI applies automatic caching with a read discount on long prompts as well. Source: Anthropic, https://docs.claude.com/en/docs/build-with-claude/prompt-caching, 2026-05-31 Source: OpenAI, https://platform.openai.com/docs/guides/prompt-caching, 2026-05-31
How it works
Caching is prefix-based. The provider checks whether the prompt up to a designated breakpoint is already cached; if so it reuses it, otherwise it processes the full prompt and caches the prefix once the response starts. Anthropic caches the full prefix in the fixed order tools, then system, then messages. There are two modes: automatic (one cache_control flag at the top level, the breakpoint moves forward as the conversation grows) and explicit breakpoints (place cache_control on specific blocks for fine control). The default cache lifetime is 5 minutes, refreshed for free on each hit, with a 1-hour option at higher write cost. Source: Anthropic, prompt-caching, 2026-05-31
The structural consequence
The ordering rule, stable prefix first, volatile content last, becomes a design constraint. Put the system prompt, tool definitions, and large static context (a codebase, a document) at the front where they cache, and keep the changing user turn at the end. Reordering content to break the prefix silently invalidates the cache and the savings evaporate. This is the cost-side complement to Context Engineering: the same "stable rails up front" layout that helps the model also helps the cache.
Why it matters for agents
Agents are the ideal caching workload: long multi-turn loops over a fixed system prompt and tool set, exactly the "repetitive tasks with consistent instructions" caching targets. It also changes the math on expensive architectures, a multi-agent system that burns ~15x the tokens of a chat is far more viable when the shared lead-agent prefix is cached. Caching is therefore a load-bearing assumption in any Services-as-Software unit-economics model. Source: Anthropic, prompt-caching, 2026-05-31
Cache-aware context layout
| Context segment | Cache stance |
|---|---|
| System/developer instructions | stable prefix; keep early and rarely change |
| Tool schemas | stable prefix; avoid reordering casually |
| Large static docs/code excerpts | cache when reused across turns or workers |
| Retrieved search results | volatile; put after stable rails |
| User turn and current diff | volatile; put last |
| Agent scratch/plan state | volatile unless persisted outside context |
This is why Context Engineering and Performance Budgets are tied together. Context layout affects quality, latency, and cost at the same time.
Failure modes
- Reordering tool definitions breaks cache locality.
- Stuffing volatile retrieval before stable instructions invalidates the prefix.
- Per-worker prompts in multi-agent systems diverge enough that shared cache benefits vanish.
- Cached context becomes stale when the source of truth changes but the harness keeps reusing it.
- Teams treat caching as a billing feature instead of an architecture constraint.
Concept Position
| Field | Value |
|---|---|
| Concept family | Brain, memory, and retrieval |
| Concept owned | Prompt caching lets the provider reuse computation over a repeated prompt prefix, cutting both cost and latency on cache hits by roughly an... |
| Category map | Concept System Map |
Timeline
- 2026-07-01 | Concepts category refresh added this page to the Brain, memory, and retrieval family, linked it to Concept System Map, and kept it standalone because it owns this reusable mental model: Prompt caching lets the provider reuse computation over a repeated prompt prefix, cutting both cost and latency on cache hits by roughly an... Source: User request, 2026-07-01
- 2026-06-19 | Added cache-aware context layout and failure modes, linking prompt caching to performance budgets. Source: whole-wiki concept review, 2026-06-19
- 2026-06-19 | Linked Agent Unit Economics as the broader cost-per-successful-outcome model that prompt caching feeds into. Source: whole-wiki concept review, 2026-06-19
- 2026-05-31 | Page created from Anthropic's prompt-caching docs (cache read ~0.1x, write 1.25x/2x, prefix order tools/system/messages, 5-min default, automatic vs explicit breakpoints) and OpenAI's automatic-caching guide. Tied the stable-prefix-first layout to Context Engineering and the cost of Multi-Agent Topologies. Source: https://docs.claude.com/en/docs/build-with-claude/prompt-caching, 2026-05-31