1:1 mentoring with Big Tech AI engineers
System Design

Agent Tech Stack

Ten components, one default and one trigger each: what to buy and what to own, the same parts with current cloud names, the model door, the gateway, the cost at three sizes and the one trace-sampling policy the other pages rely on.

Last updated

Production12 min readFirst readDeploy Your First AgentThe First 24 Hours

After this section you can

  • Name a default, the criterion that decides it and the trigger that changes it, for each of ten components
  • Map the same components to current cloud services without a per-cloud table for each one
  • Derive a per-run model cost from calls, tokens and the current prices, and say what it excludes
32

Agent Tech Stack

Ten components, one default and one trigger each: what to buy, what to own, and what one run costs at three sizes.

Key idea

Buy the parts whose semantics are hard and general, own the loop, the prompts, the state schema and the eval cases, and add a component only when a measured trigger forces it.

Ten components on one request path: own or build four, buy six
REQUEST PATHONE POSTGRESACROSS EVERY COMPONENT RequestQueue and busRuntimeAgent loopGatewayModel accessStateVector and knowledge ObservabilityEvalsSecrets and CI/CD BUYBUYBUYBUYBUYBUY OWNBUILDOWNOWN a call to your APIcloud managed queuea managed container platform a loop you wrote,with checkpointsyour prompts live here too in-process, six jobs:keys · routing · budgetsretries · fallback · accounting a provider API,one pinned model ID the run ledger: a steps tablepgvector, beside the ledger checkpointretrieve own or build itbuy the defaultthe model (bought) Each default holds until a measured trigger fires; the table below names the trigger for each box. spans from every hop, tail-sampledgate every release of loop and promptsmodel keys live in one place: the gateway OpenTelemetry and one trace viewerJSONL cases and a runner in CIthe cloud secret store and your CI

Each box: a component and its default. Pans on a phone.

Related

More in System Design

Get full access to all 149 sections with code examples, diagrams, and interactive animations.

Unlock Premium