How to Answer
“It is a prefix cache. The provider hashes the front of your prompt, and if it matches something it has already processed it reuses the computed attention state instead of recomputing it. Two things follow from the word prefix: it only works from the very start of the prompt, and it breaks the moment any byte before the cached point changes.
So the whole game is prompt layout. Stable things first — system prompt, tool schemas, few-shot examples, the long document. Volatile things last — the user’s message, the timestamp, anything personalised.
The classic mistake is rendering ‘today is 2026-08-03 14:32:11’ or the user’s name into the top of the system prompt. That is a miss on every single call, and nobody notices, because the system still works — it is just several times more expensive than it should be.
Caveats: there’s a minimum cacheable length, a cache write costs a little more than a normal token, and the TTL is minutes. So it pays on repeated traffic, not on a long tail of one-off requests.”