MCP in Production
Run a stateless MCP server as a fleet: what replicas still share, limits, health and timeouts, why a lost stream means a repeated request, failing so the model can recover and you can see it, and changing a tool without breaking callers.
Last updated
After this section you can
- Name what replicas must still share, and the store each one needs
- Choose the error channel and the idempotency key for each tool
- Plan a tool change that replicas on both versions can serve
MCP in Production
Run a 2026-07-28 MCP server as several replicas: what they share, how calls repeat and fail, and how to change a tool safely.
Any replica can answer any request, so scaling is a load balancer and a few stores. What breaks is what the replicas still share: a secret key, a notification bus, a task store, and the retry a lost response makes the client send again. Make repeated writes safe, make failures visible, and change tools so older clients keep working.
Blue is the request path, teal the replicas, purple the state they share. The figure pans sideways on a phone.