Question 1

What's the difference between a chatbot and an agent?

Accepted Answer

"A chatbot is a single LLM call — input in, text out, stateless. An agent is an LLM inside a loop. The loop gives it tools, memory, and the ability to take actions in the world. The agent decides what to do next based on observations. The key difference is autonomy — an agent can reason, act, observe, and iterate until a task is complete. The 'agent' is actually the while-loop your code runs around the LLM, not the LLM itself."

Question 2

When would you NOT use an agent? When is a simple RAG pipeline enough?

Accepted Answer

"If the task is single-turn retrieval + generation — user asks a question, you find the answer in docs — RAG is cheaper, faster, and more predictable. I'd reach for agents only when: (1) the task requires multiple steps (2) it needs tool use (write operations, calculations, API calls), or (3) the solution path is not known upfront and requires reasoning. An agent adds 3-10x the cost and latency of RAG. The tradeoff is autonomy vs predictability."

Question 3

How do you decide between ReAct, Planner-Executor, and Multi-Agent?

Accepted Answer

"Decision tree: ReAct when the task is exploratory and the path isn't known upfront (research, diagnosis). Planner-Executor when the task has clear phases and I want auditability — the plan is a human-readable artifact I can approve before execution. Great for compliance-heavy workflows. Multi-Agent only when there are genuinely separable domains of expertise — e.g., a researcher who searches the web and an analyst who runs SQL shouldn't share a context window. Multi-agent is a tool, not a defau…

Question 4

Your agent is taking 15 tool calls to complete a task that should take 3. What do you do?

Accepted Answer

"I'd diagnose in this order: (1) Check tool descriptions — are they ambiguous? If the agent can't tell which tool to use, it tries them all. Fix the descriptions. (2) Check if it's re-calling the same tool — add deduplication. (3) Check context bloat — after 10 calls, the context is so long the model loses track. Add observation summarization. (4) Consider a planner step — if the agent is wandering, a plan upfront constrains the path. (5) Check the system prompt — add explicit guidance like 'you…

Question 5

When do you fine-tune instead of adding retrieval or changing the prompt?

Accepted Answer

"Three levers, and they fix different failures. Prompting fixes instruction-following — the model can do the task, it just isn't doing it your way. Retrieval fixes missing knowledge — the model doesn't know your data, or your data changes daily. Fine-tuning fixes consistency — the model knows how, but drifts on format or tone across thousands of calls, or you need a smaller model to hit a latency and cost target. The order matters: prompt, then retrieval, then fine-tune. Most teams reach for fin…

Question 6

Walk me through your RAG pipeline. What chunk size and overlap, and why?

Accepted Answer

"Six stages — chunk, embed, index, retrieve, rerank, generate. The interesting decisions are at the two ends. Chunking: I start at about 512 tokens with 10–15% overlap for prose, but I don't chunk structured documents on a token count at all. I chunk them on their own boundaries — a section, a whole table, a function. The failure everyone hits once is splitting a table so that neither half means anything. Retrieval: dense alone misses exact strings — part numbers, error codes, names. So hybrid.…

Question 7

How do you get reliable structured output? What happens when the JSON doesn't parse?

Accepted Answer

"First, don't ask for JSON in prose and hope. Use the provider's constrained decoding — attach a schema to the request so invalid tokens are masked at sampling time and the output parses by construction. That buys you syntactic validity, not semantic validity. The model will happily return a perfectly-shaped object with a date in the wrong format, an enum value it invented, or line items that don't add up to the total it also gave you. So there are two layers: constrained decoding for shape, the…

Question 8

When would you not use an agent framework and write the loop yourself?

Accepted Answer

"The agent loop is about forty lines — call the model, if it asked for tools run them, append the results, call again, stop on a final answer or a step cap. That's not the hard part, and adopting a framework doesn't remove the hard part. What a framework does give you is the machinery around the loop: retries, streaming, tracing, checkpointing, human-in-the-loop pauses, durable state. Writing all of that yourself is real work, and that's the honest argument for using one. What it costs you is a…

60 Interview Questions & How to Answer