1:1 mentoring with Big Tech AI engineers
LLM & Agentic

Prompt Caching

Cut Claude cost and latency by up to 90% with prompt caching: cache the stable prefix (system prompt, tools, documents, history), pay the write premium once, then read for a fraction of the input price (0.1x on most models, 0.05x on Opus 5.5 and Sonnet 5.5).

Last updated

Production11 min readFirst readMessages APITokens, Context Windows & Sampling

After this section you can

  • Place cache breakpoints so the stable prefix is cached and the volatile tail is not
  • Work out write-versus-read costs per model and choose a TTL
  • Find silent invalidators and prove caching works from the usage fields
25

Prompt Caching: Pay Once for the Prefix

Every call resends tools, system prompt and history, and you pay to process them again. Prompt caching lets later calls read that repeated prefix at a fraction of the price and with less wait, as long as its bytes do not change.

Key idea

Mark where the stable part of your prompt ends with cache_control. The first call writes that prefix at a small premium; calls in the next few minutes that start with exactly the same bytes read it at a fraction of the price: 0.1× on Haiku 5.5 and Opus 5, 0.05× on Sonnet 5.5 and Opus 5.5, 0.025× on Fable 5.1. Change one byte and everything after it is a miss.

The cache is a prefix: one changed byte invalidates everything after it
RENDER ORDER IS CACHE ORDER tools fixed, sorted by name system frozen: no dates, no IDs messages: earlier turns append-only, never edited new user turn different every call breakpoint 1 breakpoint 2 NEXT CALL · SAME PREFIX read from cache: 0.1× input price or less full price NEXT CALL · A TIMESTAMP ADDED TO THE SYSTEM PROMPT still read changed at the system prompt: everything from here is written again at 1.25× Stable content first, volatile content last, breakpoints on the boundaries between them.

Related

More in LLM & Agentic

Get full access to all 149 sections with code examples, diagrams, and interactive animations.

Unlock Premium