1:1 mentoring with Big Tech AI engineers
Q75Premium

You upgraded the model and requests started returning 400. What changed, and how do you upgrade safely next time?

Hard / Curveball

MigrationProductionStructured OutputDeployment

Commonly asked at Anthropic · Cursor · Perplexity

How to answer — what you would actually say

“A 400 right after an upgrade almost always means the request uses something the new model no longer accepts. The model ID isn’t a config value; changing it is an API change. And a 400 isn’t retryable, so if our client retried it, we just tripled the errors.

On the current models, the usual suspects are four things older code relies on: temperature, top_p or top_k at a non-default value; thinking with budget_tokens, replaced by adaptive thinking plus output_config.effort; assistant prefill, ending the messages with a partial assistant turn; and forced tool_choice, any or a named tool, which errors on Sonnet 5.5, Opus 5.5 and Fable 5.1 but not on Haiku 5.5. On the newest models, editing an earlier turn can also return a 400, because history has to be append-only.

The fixes are mechanical: drop the sampling parameters and steer with effort, use structured outputs instead of prefill or a forced tool, and manage long history with compaction or context editing.

Next time, I’d treat the upgrade as a release. Pin the model ID in every environment, read the migration guide, and run a contract test that sends every real request shape to the new model in CI. Then a dual run of the eval suite with effort set explicitly, because defaults differ: Opus 5.5 defaults to medium where Opus 5 was high. Then a canary behind a flag, with rollback being one change to the pinned ID.”

The deep dive — diagrams, tradeoff tables, and the follow-up trap

Loading…