1:1 mentoring with Big Tech AI engineers
System Design

Multi-Agent in Production

Four ways to run several agents, the named forces that justify a separate deployment, and why the default is one process.

Last updated

Production11 min readFirst readHierarchical DelegationDeploy Your First Agent

After this section you can

  • Choose one process or separate deployments, and name the force behind the choice
  • Work out the latency and availability of a chain of agents, then set its deadlines
  • Pick a wiring for a workload, and name what that choice makes you own
33

Multi-Agent in Production

Four ways to wire the same four agents, the named force that justifies a network hop, and why the default is one process.

Key idea

Agent-to-agent calls over a network are slow, costly and hard to verify, so every separate deployment has to earn its hop with a named force. Start with one agent in one process. Once a call leaves that process, contracts, budgets and traces are the design.

Same four agents, four wirings: each wire you add is a failure mode
IntakeExtractorReviewerReporterA · ONE PROCESSB · FOUR SERVICESC · ENGINE AND ACTIVITIESD · BUS AND CONSUMERScosts one failure domaineach arrow an HTTP call · costs latencycosts the enginecosts a projectionone process, one budget, one tracefour deployments: every call leaves a processa function call: cannot fail alonea network wire, arrow from the caller: timeouts, retries, duplicatesworkflow engine: the history of every stepevent bus: topics, barrier count, run_events

Amber: the agents. Purple: state the wiring keeps. Teal: the wires, the subject of this lesson. Pans sideways on a phone.

Related

More in System Design

Get full access to all 149 sections with code examples, diagrams, and interactive animations.

Unlock Premium