30 Interview Questions & How to Answer

"If the task is single-turn retrieval + generation — user asks a question, you find the answer in docs — RAG is cheaper, faster, and more predictable. I'd reach for agents only when:

• (1)the task requires multiple steps
• (2)it needs tool use (write operations, calculations, API calls), or
• (3)the solution path is not known upfront and requires reasoning. An agent adds 3-10x the cost and latency of RAG. The tradeoff is autonomy vs predictability."

"I'd diagnose in this order:

• (1)Check tool descriptions — are they ambiguous? If the agent can't tell which tool to use, it tries them all. Fix the descriptions.
• (2)Check if it's re-calling the same tool — add deduplication.
• (3)Check context bloat — after 10 calls, the context is so long the model loses track. Add observation summarization.
• (4)Consider a planner step — if the agent is wandering, a plan upfront constrains the path.
• (5)Check the system prompt — add explicit guidance like 'you should need at most 5 tool calls for this type of task.'"

"Four levers:

• (1)Parallel tool calls — if tools are independent, call them simultaneously. 3 sequential x 500ms = 1.5s → 3 parallel = 500ms.
• (2)Semantic cache — if this query was asked before, serve from cache in <100ms.
• (3)Model tiering — use Flash/Haiku for the routing step, only escalate to Pro for the final synthesis. Flash is 5-10x faster.
• (4)Streaming — start sending tokens to the user while the last tool call is still running. Perceived latency drops dramatically."

"Multiple layers:

• (1)Schema exposure — don't give the agent all 500 tables. Give it a curated catalog of 15-20 relevant tables with descriptions.
• (2)Query validation — the MCP server validates every query before execution: no SELECT *, no full table scans, max rows limit, timeout after 30s.
• (3)Dry-run cost estimation — BigQuery can estimate bytes scanned before running. Reject queries that would scan > 10GB.
• (4)Per-user quotas — 100 queries/day, max 50GB scanned/day.
• (5)Row-level security — query runs as the user, not a super-account."

"Five concrete changes:

• (1)BAA — must have a signed Business Associate Agreement with every vendor in the chain (GCP, model provider). Vertex AI supports BAA.
• (2)PHI handling — all patient data redacted by Cloud DLP before any LLM call. The model never sees raw PHI.
• (3)Encryption — CMEK for data at rest, mTLS for transit, VPC-SC perimeter around the entire system.
• (4)Audit trail — every access to PHI logged with who, when, what, why. Retained 6 years.
• (5)Zero data retention — must confirm model provider doesn't retain prompts/responses for training. Vertex AI ZDR is on by default. I'd also add access reviews every 90 days and annual penetration testing."

"Common causes of quality drift:

• (1)Data drift — the knowledge base hasn't been updated. New products, pricing changes, policy updates aren't in the RAG corpus. Fix: automated re-indexing pipeline.
• (2)Usage pattern drift — users are asking questions the agent wasn't designed for. Fix: classify query types, track 'out-of-scope' rate.
• (3)Model version change — the provider silently updated the model. Fix: pin model versions, run golden set on every version change.
• (4)Prompt injection at scale — users found ways to jailbreak. Fix: review flagged outputs. Debug process: run the golden set from week 1 — if it still passes, the issue is data/usage drift, not model quality."

"Defense in depth:

• (1)Channel separation — system prompt is in the 'system' role, retrieved documents are wrapped in <document> tags in the 'user' role. The model is instructed to treat document content as data, never as instructions.
• (2)Input sanitization — scan retrieved docs for known injection patterns before including in prompt.
• (3)Output validation — a lightweight classifier checks if the response contains system prompt content, internal instructions, or out-of-scope tool calls.
• (4)Dual-LLM pattern — for high-stakes outputs, a second model reviews the first model's output for policy violations.
• (5)Behavioral testing — red-team the agent weekly with known injection attacks."

"Cost reduction playbook:

• (1)Model tiering — use Flash/Haiku for 80% of tasks (classification, simple Q&A), Pro/Opus only for complex reasoning. That alone cuts 60-70%.
• (2)Prompt caching — system prompt + tool definitions are identical across calls. Cache them. Saves 80% on input tokens for repeated calls.
• (3)Semantic caching — if someone asked a similar question in the last 24h, serve from cache. Expected 30-40% hit rate for support use cases.
• (4)Context trimming — summarize old tool observations instead of keeping full text.
• (5)Batch API — for non-urgent tasks, use the batch endpoint at 50% discount. Combined, these typically achieve 5-8x cost reduction."

"The architecture shouldn't change — the infrastructure scales.:

• (1)Stateless agents on Cloud Run/GKE — auto-scale horizontally.
• (2)Queue-based ingestion via Pub/Sub — decouples request rate from processing rate.
• (3)Provisioned throughput on Vertex AI for predictable latency under load.
• (4)Sharded vector indices — partition by tenant or region.
• (5)Regional deployment — deploy in 3 regions, route by user geography.
• (6)Cache layers become critical — semantic cache hit rate determines your cost scaling. The key insight: at 10K users you can afford to be synchronous. At 1M, everything must be async with graceful degradation."

"Three layers:

• (1)Short-term — the message history array. This IS the memory for the current conversation. It's just text appended to the prompt each turn.
• (2)Long-term — persist key facts to a vector store after each conversation. Before the next conversation, retrieve relevant past context. Example: 'User prefers formal tone' or 'User's team uses Python 3.11.'
• (3)Structured memory — for critical facts, store in a database (not just vector). User preferences, past decisions, account metadata. The challenge: what to remember and what to forget. I'd use an LLM-based summarizer at conversation end to extract durable facts, and TTL-based expiry for ephemeral ones."

"Context management strategies:

• (1)Observation summarization — after each tool call, summarize the result to 2-3 sentences instead of keeping the full response.
• (2)Sliding window — keep the system prompt + first user message + last N messages. Drop middle turns.
• (3)Context compaction — Claude's Agent SDK does this automatically: when approaching the limit, it summarizes older turns while preserving key facts.
• (4)Hierarchical delegation — offload sub-tasks to sub-agents with their own context windows. The orchestrator only sees summaries.
• (5)External scratchpad — write intermediate results to a file/DB, reference by ID instead of keeping in context."

"MCP when:

• (1)The integration will be used by multiple agents — build once, use everywhere.
• (2)You need a security boundary — the MCP server enforces auth, rate limits, and PII scrubbing independent of the agent.
• (3)You want vendor decoupling — switch from Claude to Gemini without rewriting integrations. Direct tools when:
• (1)It's a simple computation (calculator, date parsing) — MCP overhead isn't worth it.
• (2)Prototype speed — inline tools are faster to write.
• (3)The tool is agent-specific and won't be reused. Rule of thumb: if it talks to an external system, use MCP. If it's pure logic, use a direct tool."

"Tool selection degrades above ~15 tools per agent. Solutions:

• (1)Tool routing — a fast classifier (Flash) categorizes the query first, then only relevant tools (5-8) are attached for that category.
• (2)Tool groups — separate MCP servers by domain (billing tools, support tools, analytics tools). Attach only the relevant server per task.
• (3)Better descriptions — include 'when NOT to use this tool' in the description. Be explicit: 'Use get_account for account metadata. Do NOT use this for billing data — use get_billing instead.'
• (4)Few-shot examples — in the system prompt, show 2-3 examples of correct tool selection for common query types."

"Three levels of isolation, choose based on sensitivity:

• (1)Row-level security (cheapest) — shared infrastructure, data filtered by tenant_id in every query. Good for low-sensitivity SaaS.
• (2)Namespace isolation — shared compute but separate vector indices, separate KMS keys, separate logging sinks per tenant. Good for mid-tier.
• (3)Project-level isolation (strongest) — separate GCP project per tenant with its own VPC-SC perimeter, CMEK, and service accounts. Required for regulated industries. The agent's MCP tools enforce the isolation layer — the LLM has no concept of tenants. Critical: test isolation — run adversarial queries like 'show me data from tenant B' and verify zero cross-contamination."

"Full trace chain: every task gets a trace_id that links:

• (1)The original user request
• (2)The plan the agent generated
• (3)Every tool call with arguments and results
• (4)Every LLM prompt and response (stored but PII-redacted)
• (5)The final output with confidence score
• (6)Any human approval steps. This goes to BigQuery with 7-year retention. For the executive: a human-readable summary is auto-generated: 'The agent reviewed 3 data sources, found X, concluded Y, and took action Z.' Think of it as an automated decision log — every AI decision is as auditable as a human decision."

"Three evaluation strategies:

• (1)Outcome-based eval — I don't care about the path, I care about the result. Did the agent produce the correct answer? Was the customer satisfied? Use golden sets with expected outcomes and LLM-as-judge for quality.
• (2)Trajectory-based eval — for important tasks, check that the agent took reasonable steps. Did it call the right tools? Did it ask for the right data? Score the trajectory, not just the output.
• (3)A/B testing — run the new version on 10% of traffic, compare CSAT, accuracy, cost, latency vs the current version. Only promote if all metrics are equal or better. The key: separate what from how. The outcome must be deterministic (correct answer); the path can vary."

"Multi-layer:

• (1)Citation enforcement — every factual claim must cite a retrieved chunk. Claims without citations are flagged.
• (2)Claim verification — extract individual claims from the answer, check each against the source material. Score = supported claims / total claims.
• (3)Self-consistency — generate the answer 3 times with temperature > 0. If answers diverge significantly, confidence is low — flag for human review.
• (4)Programmatic checks — for numbers, dates, prices: verify against the source data directly. No LLM needed.
• (5)'I don't know' calibration — train the model to say 'I don't have enough information' when context is insufficient. Measure the rate and verify it's appropriate."

"Four areas change:

• (1)Embeddings — switch to multilingual model (e.g., multilingual-e5-large). Test retrieval quality per language — it degrades for low-resource languages.
• (2)Chunking — sentence boundaries differ. Use language-aware tokenizers. Arabic is RTL — ensure your pipeline handles it.
• (3)Evaluation — build golden sets per language. LLM-as-judge must be multilingual or use per-language judges.
• (4)Tool schemas — keep in English (models handle English schemas best). But response generation should be in the user's language. Add 'Respond in {detected_language}' to the system prompt. Cost implication: multilingual embeddings are larger; retrieval quality is usually 10-15% lower for non-English."

"Prevention layers:

• (1)Action limits — no single agent action can exceed $X without human approval. For financial actions, implement 4-eyes principle.
• (2)Idempotency — every write action has an idempotency key. Retries don't double-execute.
• (3)Reversibility — prefer reversible actions. Don't delete; soft-delete. Don't send; draft.
• (4)Insurance via audit trail — full decision trace proves the agent followed its instructions. Liability typically sits with the company that deployed the agent, not the model provider — your terms of service should reflect this.
• (5)Graceful degradation — when confidence is below threshold, the agent must route to a human, not guess. The meta-answer: the agent should never be the sole decision-maker for high-value actions."

"This is why architecture matters:

• (1)MCP servers don't change — they're model-agnostic. All tool integrations survive.
• (2)Prompts need tuning — each model family responds differently to system prompts. Budget 1-2 weeks for prompt engineering.
• (3)Evaluation is the safety net — run the golden set on Gemini, compare scores vs Claude. Only migrate when quality parity is confirmed.
• (4)Agent loop is the same — tool_use/tool_result follows the same pattern across providers with minor field name changes. The lesson: decouple your intelligence layer from your integration layer. The model is a replaceable component."

"Workflow-per-agent pattern:

• (1)Router agent — a thin classifier that receives the user request, identifies the workflow type, and dispatches to the appropriate specialist agent.
• (2)Specialist agents — each workflow has its own agent with its own system prompt, tools, and eval criteria. Deployed as separate services.
• (3)Shared infrastructure — all agents share the same MCP servers, vector store, and observability pipeline.
• (4)Configuration-driven — agent behavior defined in YAML/JSON configs, not code. New workflows added by writing a config + prompt, not deploying new code. This avoids the monolith while keeping infrastructure costs shared."

"Agent A/B testing framework:

• (1)Split by user cohort, not by request — the same user should get the same variant for consistency.
• (2)Metrics to compare: task completion rate, CSAT, cost per task, latency, hallucination rate. Need all to be equal or better, not just one.
• (3)Shadow mode first — run variant B on all traffic but only show variant A's output. Compare offline. Only promote B to live when confident.
• (4)Statistical significance — agent outputs are high-variance. Need larger sample sizes than UI tests. Typically 1000+ tasks per variant.
• (5)Prompt version tracking — every prompt change is a versioned artifact in git. A/B test maps to prompt version A vs B."

"Never in the agent, never in the prompt.:

• (1)Secret Manager — all credentials stored in GCP Secret Manager, rotated automatically.
• (2)Per-MCP service accounts — each MCP server has its own service account with minimum required permissions.
• (3)Workload Identity Federation — for MCP servers running on GKE/Cloud Run, no static keys at all. Identity is asserted by the platform.
• (4)User-scoped tokens — when the MCP needs to act as the user (e.g., read their Gmail), use OAuth with the user's delegated token, stored per-session, never persisted.
• (5)The LLM never sees credentials — it emits 'I want to call tool X with args Y.' Your dispatcher adds the credentials. Separation of intent from execution."

"This is a tone/brand problem, not an accuracy problem.:

• (1)Brand voice system prompt — define the voice: 'Professional but approachable. No jargon. Lead with the business impact, not the technical detail.'
• (2)Audience-aware formatting — detect the recipient's role. VP gets a 3-bullet summary with metrics. Engineer gets the detailed analysis.
• (3)Tone classifier — post-generation filter that scores tone (formal/casual, confident/hedging, concise/verbose). Flag outputs that don't match the target profile.
• (4)Feedback loop — VP's edits are gold training data. Analyze what they change — it's usually not the facts, it's the framing."

"Structured debugging:

• (1)Segment failures — by task type, user segment, time of day, input length. Is it 20% across the board or 80% on one category?
• (2)Read the traces — pull 20 failed traces. Classify: tool error? Model hallucination? Timeout? Wrong tool selection? Context overflow?
• (3)Find the common pattern — usually 1-2 root causes explain 80% of failures.
• (4)Fix and verify — fix the root cause, replay the failed traces, confirm they now pass.
• (5)Add regression tests — add the failed cases to the golden set so this never regresses.
• (6)Monitor — set an alert on failure rate. The meta-insight: the observability you built before production is what makes this debugging possible in 1 hour instead of 1 week."

":

• (1)A golden evaluation set — 200 tasks with expected outputs. If I can't measure quality, I can't ship safely. This is the most under-invested thing in AI projects.
• (2)A kill switch — one button that routes all requests to humans. When things go wrong (and they will), I need to stop harm instantly.
• (3)An audit trail — every decision the agent makes, with the full trace of why. For compliance, for debugging, and for the inevitable 'why did the agent do X?' question from the customer's CISO. Everything else — caching, scaling, fancy UX — can come after launch. These three are non-negotiable for responsible deployment."