Deployment & Rollout
How a running LLM system ships a model or prompt change with evidence: version manifests, mirroring and replay, canary sample arithmetic, failure-mode-first triggers, one-move rollback and the GPT-4o case.
Last updated
After this section you can
- Choose a mirror, a canary or a pointer flip for a change, and defend it on evidence and blast radius
- Derive how long a canary must hold from its traffic share, judged fraction and the drop you need to see
- Write failure-mode-first triggers and trace the GPT-4o rollback through them
22
Deployment & Rollout
Ship a model or prompt change on evidence: mirror it, let judged samples set the hold time, and undo it with one move.
Key idea
A release is a manifest with immutable versions and one movable label. Every request logs its version. Move traffic in stages that wait for judged samples (responses a judge model has scored), not the clock, and roll back by setting the label, never by toggling it.
Each rung must earn the next with evidence; a failed rung sends the label back
Amber gate, blue replay, teal canary and ramp, green soak, purple label, red abort path. Pans sideways on a phone.