5-Phase Framework
Five-phase system design framework for AI interviews: requirements, architecture, data flow, scaling, and production readiness.
Last updated
The 5-Phase System Design Framework
A repeatable skeleton for any agentic system-design scenario. It mirrors how Google FDEs run real customer engagements — and, not by coincidence, exactly what interviewers are scoring you on.
The hardest moment in a system-design interview is the first thirty seconds — a blank whiteboard and an open-ended prompt. This framework removes that moment. You always start the same way, so your energy goes into the problem, not into deciding what to do next. The five phases also front-load the parts interviewers weight most heavily: the questions you ask before you draw a single box.
Most candidates lose points not because their architecture is wrong, but because they jump straight to drawing boxes and never establish what they are building or why. A senior engineer treats the prompt as the start of a conversation, not a spec to be implemented. The framework below enforces that discipline. Each phase has a rough time budget for a typical 45-minute round — adjust to taste, but keep the order.
Phase 1 · Restate the Problem
Before anything else, play the problem back in your own words. This is not filler — it is a cheap insurance policy. Interviewers routinely leave the prompt vague on purpose, and a thirty-second restatement surfaces a mismatched assumption while it still costs nothing to fix. It also buys you a moment to think.
The trick is to add something, not to parrot. Fold in a constraint the interviewer implied but did not say, and name the scope you are committing to.
Do
“So we’re building an assistant that answers employee questions over internal docs and can take actions like filing a ticket — and it needs to stay within our data boundary. I’ll focus on the read + action path first.”
Don’t
“Okay, so you want an internal chatbot.” (Verbatim, adds nothing, and quietly narrows a rich problem down to a toy.)
Phase 2 · Clarify — the phase that wins interviews
This is where staff-level candidates separate themselves. Great clarifying questions prove you have shipped these systems, because they target the decisions that actually shape the architecture. Ask three to five, then stop — this is a scalpel, not a survey.
Agentic systems add a second layer of questions on top of the classic ones. The classic axis is about load and correctness; the agentic axis is about autonomy and trust. You need both.
| Classic system-design questions | Agentic system-design questions (add these) |
|---|---|
| What’s the scale — QPS, users, data volume? | What can the agent do without human approval? |
| What’s the latency SLA? | What’s the blast radius of a wrong action? |
| Consistency vs availability trade-off? | What’s the acceptable cost per query? |
| Who owns the data; what’s the retention policy? | How do we measure whether an answer is good? |
Think of the agentic questions in four buckets. Pulling one strong question from each bucket is usually enough to show range without stalling.
Trust & autonomy
What actions are auto-approved vs gated? What’s the blast radius of a mistake? Any compliance regime (HIPAA, SOC 2, GDPR)?
Quality & evaluation
What does “good enough” look like? What does a failure look like? Is there a human-feedback loop to improve over time?
Data & context
Where does the knowledge come from, and how fresh must it be? Does the agent need memory across sessions? How many tools?
Cost & scale
Acceptable cost per query? Concurrent users at peak? Is real-time required, or is async acceptable?
“What’s the trust boundary — what can the agent do on its own, and what needs a human in the loop?” This single question reframes the entire design. A read-only Q&A bot and an agent that can issue refunds are the same LLM with wildly different architectures around it. Answer this and half your later decisions make themselves.
Phase 3 · State Your Assumptions
You will never have every answer, and you should not wait for them. State the assumptions you are designing against, out loud, and invite correction. This does three things: it keeps you moving, it shows judgment about sensible defaults, and it makes the interviewer a collaborator who will steer you if you drift.
Keep it to the choices that actually change the design:
- Model choice — e.g. “a frontier model like Claude for reasoning, a small cheap model for routing and classification.”
- Agent pattern — a deterministic workflow where steps are known in advance, or an agent that decides its own steps. (When in doubt, prefer the workflow; it’s cheaper and easier to debug.)
- Trust level — human-in-the-loop for irreversible actions in V1, tightening later.
- Scope — English-only, one region, one tenant for the first version.
Phase 4 · Design — walk the architecture in layers
Now you draw. The mistake here is drawing a bag of boxes with no order. Instead, follow the request as it travels through the system, top to bottom, naming each layer’s job as you go. For an agentic system the layering below works for almost any prompt — it’s the same spine whether you’re building a support bot or a coding agent.
Two things make this diagram score points. First, guardrails sit before the model, not after — you validate and scrub input before it ever reaches an LLM, and you validate actions before they execute. Second, observability is drawn as a rail, not a box. It is not a feature you bolt on at the end; it wraps every layer, because in a non-deterministic system the only way to know it works is to measure it continuously.
Narrate the layers by following one request through them. That framing — “a message comes in here, gets checked here, routed here, the agent loops here, pulling context from here, calling the model through here” — is far more convincing than reciting a list.
Two terms in that diagram are worth pinning down before we move on, because interviewers will probe both. RAG (retrieval-augmented generation) means the model looks up relevant documents at request time — in this case internal docs — instead of relying on whatever it memorized during training; it is why the context engine exists as its own layer. Semantic caching means caching answers by meaning rather than exact string match, so “how do I reset my VPN?” hits the cache entry for “VPN reset steps” and skips the model call entirely. You will see both again in the worked examples below.
Phase 5 · Harden It — the non-functional requirements
A design that only handles the happy path is a junior design. Close with the four concerns that decide whether a system survives contact with production. You don’t need to solve each in depth — naming them, with one concrete tactic each, is what signals seniority.
| Concern | What to say — one concrete lever each |
|---|---|
| Security | Guardrails, PII redaction before the LLM call, prompt-injection defense, least-privilege tool access. |
| Scale | Semantic caching for repeat queries, tiered model routing, async for long tasks, shard by tenant. |
| Cost | Per-user token budgets, route easy queries to a small model, batch offline work, cache aggressively. |
| Operability | Tracing + quality evals, a canary rollout (ship to a small slice of traffic first — say 5% of users — and widen only if error and quality metrics hold) behind feature flags, and a kill switch for the agent’s actions. |
The Framework in Action
Here is the whole skeleton applied to a common prompt, compressed to show the shape. Notice how much is decided before any box is drawn.
1 · Restate: “An assistant that answers employee questions from internal docs and can file IT tickets, without data leaving our boundary.”
2 · Clarify: Trust boundary? → ticket creation is auto, ticket closure needs a human. Freshness? → docs change weekly. Cost ceiling? → a few cents per query. Quality bar? → thumbs-up rate + zero hallucinated policies.
3 · Assume: Claude for answers, a small model for routing; RAG over docs (not fine-tuning, since facts change weekly); human-in-the-loop for closing tickets in V1.
4 · Design: Slack surface → guardrails (auth + PII) → router (question vs action) → agent runtime with two tools (search_docs, create_ticket) → context engine (RAG + short-term memory) → LLM gateway with prompt caching. Observability across all of it.
5 · Harden: Cache common questions; redact PII before retrieval; cap the agent at N steps; log every tool call; canary to one team before company-wide rollout.
Worked Example 2 · “Design an agent that reconciles invoices against purchase orders”
A different domain, the same skeleton. Accounts payable (AP) — the team that pays a company’s bills — is a classic FDE engagement, and this prompt is a favorite because it forces the trust-boundary conversation immediately: a wrong action here spends real money. Watch how little the framework changes; only the nouns do.
Phase 1 · Restate: “We’re building an agent that reads incoming vendor invoices, matches them against the purchase orders and goods-receipt records in our ERP, approves the clean ones, and routes the exceptions to a human — without it ever being able to move money on its own. I’ll focus on the match-and-approve path first.” Thirty seconds, scope stated, one constraint added that the interviewer implied but never said.
Phase 2 · Clarify. Three exchanges, one per bucket that matters most here:
- You: “What’s the matching standard — 2-way (invoice vs PO) or 3-way (invoice vs PO vs goods receipt)? And what’s the blast radius of a wrong auto-approval?” Interviewer: “3-way for physical goods. A false approval pays a fraudulent or inflated invoice — that money is gone.”
- You: “On a mismatch, does the agent try to resolve it, or flag a human?” Interviewer: “Inside a small tolerance it can auto-resolve; anything bigger goes to an AP analyst with a written explanation.”
- You: “Volume and latency — how many invoices, and is real-time needed?” Interviewer: “Roughly 30,000 invoices a month via email and the ERP portal; processing within the hour is fine.”
That last answer is a gift: async is acceptable, which means batch processing, no real-time serving constraints, and cheap models — a completely different cost profile than a chat product.
Phase 3 · Assume: a deterministic workflow for extraction and matching (the steps are known in advance), with the LLM reserved for the messy parts — parsing wildly different invoice layouts and drafting exception summaries — never for arithmetic. Auto-approve only exact matches within tolerance (≤1% or $5 per line). Extraction and classification run on a small, cheap model (~$1 per million input tokens for a Haiku-class model, as of July 2026); a frontier model writes the exception narratives. The ERP stays the system of record: the agent writes recommendations, never payments.
Phase 4 · Design. Same six layers as before, re-filled for this domain:
Phase 5 · Harden. One concrete choice per concern, each with its one-line justification — this is exactly the level of depth that scores well:
- Immutable audit log of every match decision and tool call — financial auditors must be able to reconstruct why any payment was approved, months later.
- Dual control above a dollar ceiling (say $10,000) — caps the blast radius of any single wrong auto-approval.
- Vendor-master changes are human-only — edits to bank details are the top invoice-fraud vector, so the agent never touches them.
- Exception queue with confidence scores — analysts spend their time only where the system is unsure, which is where the errors actually live.
- Golden-set eval before every prompt or model change — replay a few hundred historical invoices with known outcomes so a regression is caught in test, not in the pay run.
This design is not hypothetical. Ramp’s Bill Pay product runs four AI agents for invoice coding, fraud detection, approval summaries, and payments, with ~99% OCR accuracy on line-item extraction and invoice processing 2.4× faster than legacy AP software (per Ramp, as of 2025–2026). The fraud concern in Phase 5 is equally real: Ramp cites that 76% of businesses experienced payment fraud in 2025, which is exactly why 3-way matching and human-gated vendor changes exist. Naming one real system and one real number in an interview instantly grounds your design.
The clarifying phase is where the interview is won or lost. A candidate who opens with “What’s the trust boundary, and what’s the acceptable cost per query?” signals real agentic experience in the first two minutes. A candidate who starts drawing boxes signals the opposite. Spend the cheap five minutes up front — it earns you the right to design the correct system.
Anthropic, “Building Effective Agents” (2024) — the workflow-vs-agent distinction used in Phase 3 · Google, Site Reliability Engineering (the SRE book) — non-functional thinking for Phase 5 · Alex Xu, System Design Interview, Vol. 1–2 — the classic-questions muscle for Phase 2.
Checkpoint · Run the Framework Yourself
Four reps. Say your answer out loud before opening the sample — the muscle you are building is generating the questions under time pressure, not recognizing good ones when you read them. If your answers are in the same neighborhood as the samples, you are ready for the mock interview.
Prompt 1. Interviewer: “Design an agent that books travel for employees.” Write your four clarifying questions — one per bucket.
Sample answer. Trust & autonomy: “Can the agent charge a corporate card, or only hold and recommend? What’s the dollar ceiling for an auto-booking?” Quality & evaluation: “What does a booking failure look like — wrong dates, wrong airport code, out-of-policy hotel — and how would we measure the rate?” Data & context: “Where do flight and hotel options come from — which API — and does the agent remember traveler preferences across trips?” Cost & scale: “How many concurrent travelers at peak, and what’s an acceptable cost per completed booking?” Any four questions that cleanly hit the four buckets earn full credit; the buckets matter more than the wording.
Prompt 2. Interviewer: “Your support agent answers well, but it costs $0.40 per query against a $0.05 budget. Give me three levers, in the order you’d pull them.”
Sample answer. (1) Semantic caching — a large share of support questions are near-duplicates, so serving cached answers skips the model call entirely; highest impact, lowest risk. (2) Tiered model routing — send classification and easy FAQs to a small model (~$1 per million input tokens, as of July 2026) and reserve the frontier model for genuinely hard queries. (3) Context trimming — cap retrieved chunks and conversation history, since input tokens dominate the bill. The order is the point: cheapest and safest first, and measure the cache hit rate and quality evals after each pull.
Prompt 3. Recite the six layers of the agentic request path from memory, and explain why guardrails sit before the model rather than after.
Sample answer. User surface → guardrails & input validation → orchestrator/router → agent runtime → context engine → LLM gateway, with observability and evaluation spanning all six. Guardrails come first because anything the model sees can influence it — a prompt injection validated after the model call has already done its work — and because PII must be scrubbed before data leaves your boundary for an external API. Output validation still exists, but it is the second fence, not the first.
Prompt 4. Your invoice agent auto-approves a $12,000 payment to a fraudulent vendor. Which Phase-5 hardening choices would have caught it, and at which layer does each one act?
Sample answer. A dollar ceiling with dual control caps any single auto-approval below $12,000 (agent-runtime policy). Vendor-master changes gated to humans, with out-of-band verification of bank-detail edits, blocks the classic fraud setup (guardrails + tool permissions). An anomaly check against vendor history flags a first-time payee with a fresh bank account (context engine). The immutable audit log plus alerting on first-time payees shortens the fraud window even if a payment slips through (observability rail). And a canary rollout to a small slice of invoices would have limited total exposure while the system earned trust. The pattern to notice: no single layer is trusted — the defenses overlap on purpose.
Sources
- What is 3-way matching in accounts payable? — Ramp (2-way/3-way matching, 99% OCR accuracy, 2.4× processing speed, four AI agents, 76% payment-fraud statistic, as of 2025–2026)
- Plans & Pricing — Anthropic (per-million-token model pricing, as of July 2026)