1:1 mentoring with Big Tech AI engineers
LLM & Agentic

Prompt Caching

Cut Claude cost and latency by up to 90% with prompt caching: cache the stable prefix (system prompt, tools, documents, history), pay full price once, then read at ~10%.

Last updated

Production7 min readFirst readMessages API

After this section you can

  • Place a cache breakpoint so the stable prefix is cached and the volatile tail is not
  • Do the write-versus-read economics and know when caching pays for itself
  • Audit a prompt for the silent invalidators that keep a cache from ever hitting
32

Prompt Caching: Pay Once for the Prefix

Every Claude call re-sends the whole prompt — system, tools, documents, history — and you pay to re-process it every time. Prompt caching makes Claude remember the expensive, stable part, cutting repeated-context cost by up to 90% and latency with it.

THE CENTRAL IDEA

The Messages API is stateless, so a RAG call with a 20-page document, or an agent on turn 30, re-sends and re-processes a huge identical prefix every single request. Prompt caching lets you mark that prefix with a cache breakpoint: the first call writes it to a server-side cache; every later call with the same prefix reads it for a fraction of the price and skips reprocessing it. Cache reads cost about 10% of normal input tokens. For any workload with a big, stable prefix — and almost all of them have one — this is the single highest-leverage cost and latency optimization available.

Cache the stable prefix, vary only the tail — reads cost ~10% of input
THE PROMPT, SPLIT AT A BREAKPOINT STABLE PREFIX — cache this system prompt · tool defs · big documents · few-shot examples identical across calls — the expensive part cache_control VARIABLE TAIL the new user question — tiny 1st call — CACHE WRITE prefix processed + stored cost: ~1.25× input (one time) Nth call — CACHE READ prefix skipped, reused cost: ~0.1× input · ~90% off same prefix within the TTL window TTL: 5 min (default) refreshed on each hit · 1 hour (extended) · minimum prefix: 512–4096 tokens, by model

Related

More in LLM & Agentic

Get full access to all 74+ sections with code examples, diagrams, and interactive animations.

Unlock Premium