1:1 mentoring with Big Tech AI engineers
AgenticPrep.ai
Interview PlanRoadmapLLM & AgenticRAGMCPSystem DesignInterview PrepBlogDiscussAI System Design Studio
Q8

When would you not use an agent framework and write the loop yourself?

Fundamentals & Architecture

ArchitectureAgentsTool UseProduction

Asked at Anthropic · Cursor · Sierra

How to Answer

“The agent loop is about forty lines — call the model, if it asked for tools run them, append the results, call again, stop on a final answer or a step cap. That’s not the hard part, and adopting a framework doesn’t remove the hard part.

What a framework does give you is the machinery around the loop: retries, streaming, tracing, checkpointing, human-in-the-loop pauses, durable state. Writing all of that yourself is real work, and that’s the honest argument for using one.

What it costs you is a layer between you and the model. When tool selection goes wrong you need to see the exact request that went over the wire, and that’s harder through three abstractions — especially when a version bump quietly edits the prompt underneath you.

So: own the loop and own the prompt, adopt libraries for the plumbing. If a framework won’t show me what the model was actually sent, that’s disqualifying.”

The deep dive — diagrams, tradeoff tables, and the follow-up trap

What is actually in the loop

The loop is about forty lines. The machinery around it is the real work.
What is actually in the loop 1 call model messages + tool schemas 2 tool calls? none → return the answer 3 execute your timeouts, your retries 4 append results go back as messages Step cap of 8. Exceeding it raises — it is a bug to investigate, not a condition to retry. Everything hard lives in step 3 and in the prompt that shapes step 1.

Decide per concern, not per framework

This is not an all-or-nothing choice, and treating it as one is how teams end up rewriting a workflow engine badly.

ConcernDefaultWhy
Loop control and step capownforty lines, and it is where your policy lives
System prompt, tool descriptionsownthis is the product — no upgrade should edit it
Retries, backoff, rate limitsadoptsolved, boring, and easy to get subtly wrong
Tracing and spansadoptyou want OpenTelemetry, not a bespoke logger
Durable state, resume after crashadopta real workflow engine beats a status column
Human-in-the-loop pausesplitadopt the mechanism, own the policy that triggers it

The loop, in full

def run(messages, tools, max_steps=8):
    for _ in range(max_steps):
        reply = model.call(messages, tools=tools)   # your prompt, your schemas
        messages.append(reply)
        if not reply.tool_calls:
            return reply.text
        for call in reply.tool_calls:
            messages.append(execute(call))          # timeouts and retries here
    raise StepBudgetExceeded(max_steps)             # never spin forever
REAL SYSTEM

A support agent on an off-the-shelf framework was picking the wrong tool about 9% of the time and nobody could say why — the framework was injecting its own tool-choice preamble ahead of ours. Dropping to a hand-written loop (~60 lines) made the request inspectable, and the actual fix was one sentence in a tool description. Selection errors fell to ~2%. Retries, tracing and the workflow store all stayed on libraries; the loop and the prompt came in-house.

FOLLOW-UP TRAP

“So frameworks are bad?” — no. The bad version is adopting one so you don’t have to understand the loop. If you can’t draw what goes over the wire on each step, the framework is a liability the first time production misbehaves. Teams that understand the loop use frameworks well, and switch off the parts that fight them.

Previous

How do you get reliable structured output? What happens when the JSON doesn't parse?

Next

Latency SLA is 2 seconds but your agent needs 3 tool calls. How do you meet it?