The Paradigm Shift
Traditional vs agentic system design: the 7 dimensions that transform, anatomy of an agentic system, control flow paradigms, failure modes, and when to go agentic.
Last updated
Agentic System Design: The Paradigm Shift
Traditional system design meets AI autonomy. The primitives you already know don't disappear — they transform once your system can plan, act, and adapt on its own.
Agentic system design is not a new bag of components. It is the same primitives — load balancers (which spread incoming requests across a pool of servers), caches (fast in-memory copies of recent results so you skip recomputing them), queues (waiting lines that buffer work so a slow step never blocks a fast one), retries — re-shaped by two facts that change everything: the system is non-deterministic (the same input can take different paths), and it is autonomous (it decides its own next step within the boundaries you set). Master how each familiar concept transforms, and agentic design stops feeling exotic.
In a traditional service, you write every path. A request comes in, flows through code you control, and a deterministic response comes out. You can unit-test it because the same input always yields the same output. An agentic system breaks that contract on purpose: you hand the model a goal and a set of tools, and it loops — reasoning about what to do, taking an action, observing the result, and deciding again — until it judges the task done. Your job shifts from scripting steps to designing boundaries for autonomous action.
The Seven Dimensions That Transform
Interviewers rarely ask “what is an agent?” They ask you to design one, then watch whether you understand how each classic concern changes shape. Here are the seven that come up most. Read each row as “the thing you know” becoming “the thing you now have to design for.”
| Dimension | Traditional | Agentic | Why it matters |
|---|---|---|---|
| Request flow | Request → process → response | Think → act → observe → repeat | A single call becomes a multi-step loop with dynamic re-planning. |
| Determinism | Same input → same output | Same input → varied paths | You evaluate on a rubric, not with assertEquals. |
| State | DB + session + cache | Working, episodic, semantic & procedural memory | The context window is the working memory — and it is finite. |
| Duration | Milliseconds to seconds | Seconds to hours | The SLA becomes completion quality and a cost ceiling, not p99 latency. |
| Failure | Retry → fallback → circuit breaker (stop calling a service that keeps failing) | Reflect → re-plan → re-execute | The agent reads its own errors and adapts strategy. |
| Control | Developer writes every path | Bounded autonomy | You design guardrails, budgets, and kill switches — not if/else. |
| Testing | Unit + integration tests | Evaluation: LLM-as-judge, human eval, benchmarks | Quality is scored, not asserted; regressions are statistical. |
Three Control-Flow Paradigms
The single most important architectural decision in an agentic system is: how much control does the LLM have? There are three answers, and picking the right one is most of the battle. A useful definition first: a workflow is code that orchestrates LLM calls along a path you fixed in advance; an agent lets the LLM choose its own path at runtime.
Anthropic, OpenAI, and Google all give the same advice: begin with a workflow, and add agent autonomy only where the task genuinely needs it. Autonomy buys flexibility at the price of cost, latency, and debuggability. The hybrid pattern — a deterministic skeleton with an agent node for the one genuinely open-ended sub-task — is the production default. Reach for a full autonomous agent last, not first.
When Do You Actually Need an Agent?
“Use an agent” is the wrong reflex. Most tasks are better served by something simpler and cheaper. Walk this decision tree out loud in an interview — it shows you optimize for the problem, not for using the shiny tool.
New Failure Modes You Must Design For
Traditional systems fail in familiar ways — a timeout, a 500, a full disk. Agentic systems add a whole new taxonomy of failure, because the thing making decisions is non-deterministic and runs in a loop. Name these in an interview and pair each with its mitigation; that pairing is what signals production experience.
| Failure mode | What happens | Mitigation |
|---|---|---|
| Infinite loop | Agent retries forever, oscillating between two bad states. | Hard step budget + auto-escalate to a human. |
| Context overflow | Tool outputs flood the context window; the agent loses the goal. | Context compaction + running summaries. |
| Goal drift | Over many steps the agent wanders off the original objective. | Re-inject the goal each step; checkpoint progress. |
| Tool misuse | Wrong tool, bad arguments, or an action beyond its scope. | Action classification + permission tiers (below). |
| Cost explosion | A runaway loop can burn $50+ in tokens in minutes. | Per-task token ceiling + per-user daily budget + real-time alerts. |
Guardrails: Classify Every Action
In a traditional system, the developer’s code decides what is allowed. In an agentic system, guardrails take over that job: before an action executes, you classify it by how much damage it could do, and gate it accordingly. This one table is the backbone of safe autonomy.
| Tier | Examples | Policy |
|---|---|---|
| Read-only | Search, retrieve, list, describe, read a file | Auto-approve |
| Reversible | Create a draft, write a file, update a record, add a comment | Agent decides (logged) |
| Irreversible | Delete data, send email, push code, make a payment | Human approval required |
| External | Third-party API, post to Slack, modify infra, deploy | Human approval + audit trail |
The Translation Table
Every concept you already know has an agentic counterpart. Keep this map in your head; in an interview, reaching for the right analogy (“a model router is the agentic load balancer”) instantly shows you are building on solid systems fundamentals rather than treating agents as magic.
| Traditional concept | Agentic equivalent | What to say in the interview |
|---|---|---|
| Load balancer | Model router / intent classifier | “Route simple queries to a fast model, complex reasoning to a frontier model.” |
| Database | Memory system (vector DB + KV store + graph) | “Working memory lives in context; long-term memory in a vector store.” |
| API gateway | Guardrails layer | “Every request passes input validation, PII detection, and injection defense.” |
| Message queue | Task queue + checkpoint store | “Long tasks go async; each step checkpoints so we can recover.” |
| Circuit breaker | Step budget + kill switch | “The agent gets N steps; at the limit it escalates to a human.” |
| Cache | Semantic + prompt + context cache | “Similar queries hit the semantic cache; system prompts use prompt caching.” |
| Retry logic | Self-correction loop | “The agent reads the error, reflects, and tries a different approach.” |
| Microservices | Multi-agent system | “Each agent is a specialist with its own tools; an orchestrator delegates.” |
| Health check | Eval pipeline + LLM-as-judge | “We score responses for relevance, groundedness, and safety before returning.” |
When asked to design an agentic system, open with the trust boundary: “Before I design, what can the agent do autonomously versus what needs human approval? That decides whether I reach for a workflow, an agent, or a hybrid.” Then reason down the decision tree out loud. Leading with autonomy and control — not with boxes — is the clearest signal of staff-level thinking.
Anthropic, “Building Effective Agents” (2024) — the workflow/agent/hybrid framing and “start simple” guidance · OpenAI, “A Practical Guide to Building Agents” (2025) · Google, Site Reliability Engineering — for the non-functional and failure-mode mindset.
Checkpoint: Test Yourself
Answer each out loud before opening the model answer — the interview is a speaking exam, not a reading one.
1. Walk me through the seven dimensions that transform when a system becomes agentic. Which two would you design for first, and why?
Model answer: Request flow (single pass → think/act/observe loop), determinism (same output → varied paths, so rubric-based evals), state (DB/session → working, episodic, semantic, and procedural memory), duration (milliseconds → seconds-to-hours, so SLAs become quality and cost ceilings), failure (retry → reflect and re-plan), control (scripted paths → bounded autonomy with guardrails and kill switches), and testing (assertions → scored evaluations). Most teams design for state and failure first: the finite context window and the non-deterministic loop are where production incidents actually come from. Any two are defensible if you justify them with a concrete failure story.
2. Name the five agentic failure modes and give one concrete mitigation for each.
Model answer: Infinite loop — hard step budget with auto-escalation to a human. Context overflow — context compaction and running summaries so tool output doesn’t drown the goal. Goal drift — re-inject the original objective at each step and checkpoint progress. Tool misuse — classify every action (read-only / reversible / irreversible / external) and gate by permission tier. Cost explosion — per-task token ceiling plus a per-user daily budget with real-time alerts, because a runaway loop can burn $50+ in minutes.
3. A customer-support pipeline classifies each ticket, retrieves the matching policy, drafts a reply, then a human approves it. Workflow or agent? Apply the decision tree.
Model answer: Workflow — specifically a hybrid at most. The steps are fixed and known in advance, and there are no multi-step decisions based on intermediate results, so the first branch of the decision tree (“multiple decisions?”) answers NO. Classification, retrieval, and drafting are three chained LLM calls with deterministic glue between them; the human-approval step maps to the “irreversible/external” guardrail tier. You would only add an agent node if one sub-task turned genuinely open-ended — e.g., “research this customer’s issue across five systems” — and even then it sits inside the workflow, not in place of it.