“I split the replay into two kinds. Replaying a model call costs tokens. Replaying a tool can refund a customer twice. And every step has a window between the side effect and the record of it that I can’t close outside one database transaction — so the design is about making a replay inside that window a no-op.
Three rules. Save each assistant turn, verbatim, before running any tool in it. Key every side effect by the run id plus the tool_use id — stable on every replay because that turn was saved first — and pass it to the API, or check before writing if the API has no keys. And keep the transcript append-only in a run ledger with a lease, so exactly one worker can pick the run up.
Resuming is then the ordinary loop, started from the stored transcript. The last message tells me where I died: a tool call with no result gets re-run with the same key, a finished set of results means call the model, a final answer means stop. A started call to a tool that can’t take a key goes to a person.
And I persist the step and dollar counters with the transcript, or every crash hands the run a fresh budget.”