1:1 mentoring with Big Tech AI engineers
LLM & Agentic

Agent Durability

Make a long agent run survive a crash: the window where a side effect lands before anything records it, idempotency keys that survive replay, and what to checkpoint.

Last updated

After this section you can

  • Locate the window in every loop step where a crash duplicates a side effect, and explain why it cannot be closed
  • Derive idempotency keys that survive a replay, and spot the timestamp that silently defeats them
  • Persist the transcript rather than derived state, and account for the cold cache and reset budgets a resume costs
18

Agent Durability

A long agent run is a distributed transaction nobody designed as one. When it dies at step 40, the useful question is not “can I resume?” but “what did step 39 already do to the world?”

THE CENTRAL IDEA

The model is stateless: replaying a request just costs you tokens. Your tools are not. Every run therefore has a window in each step where a side effect has landed but nothing durable records that it did — and a crash inside that window is the only one that hurts. Durability is the work of making that window small, and making a replay inside it harmless.

One step of the loop — where a crash actually costs you something
ONE STEP 1 · LOAD STATE the transcript so far 2 · MODEL CALL replaying costs tokens only 3 · RUN THE TOOL the world changes here 4 · RECORD + COMMIT now the log knows THE GAP the effect has landed, nothing records it CRASH HERE → WHAT REPLAY DOES in 1 or 2 — harmless Nothing outside the process moved. Replay costs one model call. in the gap — the real one The refund already went out. Replay sends it twice. after 4 — harmless The result is in the transcript. Resume reads it and moves on. You cannot close the gap — two writes are never one write. You can only make the replay inside it a no-op.

Related

More in LLM & Agentic

Get full access to all 74+ sections with code examples, diagrams, and interactive animations.

Unlock Premium