Graph Engineering — Designing What the Agent Is Allowed to Do Next

A loop asks the model “what should I do next?” on every turn. A graph writes down the moves that are allowed and lets the model choose inside them — and that single change is what turns a demo agent into something you can debug, resume, and trust with a refund.

The first support agent I put in front of real customers issued the same $80 refund twice. Not because the model was dumb — because the run crashed after the refund call and the retry started from the top, re-read the ticket, reached the same conclusion, and paid out again. There was no record of where it had got to, because there was no “where”: the whole agent was a while loop with a list of tools, and its only memory of progress was a message history that the retry threw away.

The fix wasn’t a better prompt or a bigger model. It was drawing the thing. Once the agent was an explicit graph — classify, check, decide, gate, execute — two of the three holes closed on their own: execute became a node you can only reach through gate, and a crashed run picked up at the node it had finished instead of at the top. The third hole — making the payout itself safe to replay — stayed my job, and I’ll come back to it. But it went from a mystery to one bounded problem with a name.

That practice has a name now too: graph engineering. It’s the discipline of designing an agent’s control flow as an explicit graph — nodes that do work, edges that decide what runs next, and one state object that threads through both — instead of letting the flow emerge from whatever the model felt like doing this time.

Chain, loop, graph

Three shapes cover almost every agent people ship. They’re not competing philosophies — they’re different amounts of structure, and the right one depends on how much of the flow you actually know in advance.

The three parts

Every graph framework is built on the same three ideas. The code here is LangGraph; learn the ideas and the API is a detail — there’s a survey of what else implements this shape near the end.

Here’s the whole idea in twenty-odd lines — a ticket comes in, gets classified, gets answered:

from typing import TypedDict
from langgraph.graph import StateGraph, START, END

class State(TypedDict):
    ticket: str
    intent: str
    reply: str

def classify(state: State) -> dict:
    return {"intent": intent_of(state["ticket"])}     # LLM call, or a rule

def answer(state: State) -> dict:
    return {"reply": draft(state["ticket"], state["intent"])}

builder = StateGraph(State)
builder.add_node("classify", classify)
builder.add_node("answer", answer)
builder.add_edge(START, "classify")
builder.add_edge("classify", "answer")
builder.add_edge("answer", END)
graph = builder.compile()

graph.invoke({"ticket": "where is order A-91?"})
# {'ticket': '...', 'intent': 'tracking', 'reply': 'It shipped Tuesday...'}

That’s still a chain — a straight line. It becomes a graph the moment one edge starts making a decision.

The engineering lives in the edges

Nodes are the boring part. Anyone can write a function that calls a model. The design work — the part that decides whether your agent is trustworthy — is in what you allow to follow what.

def where_next(state: State) -> str:
    # a pure function of state: no model call, no I/O, trivially testable
    if state["intent"] == "refund":   return "lookup_order"
    if state["intent"] == "tracking": return "shipping"
    return "escalate"

builder.add_conditional_edges(
    "classify",                                   # the node we are leaving
    where_next,                                   # what decides
    ["lookup_order", "shipping", "escalate"],     # where it may go
)

That third argument is the map from what the function returns to where the run goes. Return node names, as here, and a plain list is enough. Return abstract labels instead — "too_big", "ok" — and you pass a dict that maps each label to a node, which is worth doing once the label and the node stop meaning the same thing.

Now look at what you got for free. Routing is a function you can run over 200 past tickets in a unit test, with no model in the loop. A reviewer can see, in a diff, that escalate was added as the default. And refund can no longer happen from anywhere — there is exactly one path to it, and it goes through the gate.

Four patterns you’ll use again and again

Nearly every production graph is a mix of these four. They’re worth knowing by name, because once you see the shape you stop reinventing it.

The fan-out reducer is the one that bites people, so it’s worth seeing:

import operator
from typing import Annotated, TypedDict

class State(TypedDict):
    findings: Annotated[list, operator.add]   # concatenates: parallel-safe
    risk:     str                             # no reducer: last write wins

for check in ("fraud", "stock", "policy"):
    builder.add_edge("split", check)          # all three start together
    builder.add_edge(check, "decide")         # decide runs once, after all three

def fraud(state: State) -> dict:
    return {"findings": ["fraud: clean"]}     # appended, not overwritten

And the guard on a cycle. The counter is the whole trick, so the node that retries is the node that has to increment it:

def draft(state: State) -> dict:
    return {"reply": write(state), "attempts": state["attempts"] + 1}

def after_verify(state: State) -> str:
    if state["passed"]:          return "done"
    if state["attempts"] >= 3:   return "human"     # give up honestly
    return "retry"

builder.add_conditional_edges("verify", after_verify,
    {"done": END, "retry": "draft", "human": "escalate"})

Durability falls out for free

Here’s the part that fixed my double refund. Because the graph knows what a “step” is, the framework can save the state after every node — a checkpoint — keyed by a thread id you choose. Crash after refund and the retry resumes after refund, not from the top.

The same mechanism gives you the human checkpoint: pausing is just a checkpoint that nobody has resumed yet.

from langgraph.types import interrupt, Command
from langgraph.checkpoint.postgres import PostgresSaver

def approval_gate(state: State) -> dict:
    decision = interrupt({                      # whatever your UI needs to show
        "action": "refund", "order": state["order"], "amount_usd": state["amount"],
    })
    return {"approved": decision == "approve"}

with PostgresSaver.from_conn_string(DB_URL) as cp:
    cp.setup()                                  # creates the tables, once at deploy
    graph = builder.compile(checkpointer=cp)    # without this it can pause, never resume

config = {"configurable": {"thread_id": "ticket-8412"}}
graph.invoke({"ticket": "refund my order A-91"}, config)
graph.get_state(config).next          # ('approval_gate',) - parked, nothing lost

# ...two hours later, a different process, same thread_id
graph.invoke(Command(resume="approve"), config)

Worked example: the refund graph

Put the four patterns together and you get something that looks like a real system, because it is one. Same task as the opening story — refund a customer — drawn instead of improvised.

Four things in that drawing are what separate it from a diagram someone sketched once and left behind:

When not to reach for a graph

Graphs are not free. You pay in a state schema, a framework, and a mental model your teammates have to learn. Skip it when:

• It’s genuinely a straight line. Extract → summarize → post has no branches. Write the three function calls.
• The steps really aren’t knowable. Open-ended research is a loop with good tools. A graph whose every edge says “ask the model” is a loop wearing a costume — and a slower one.
• You’re still finding the shape. Prototype as a loop, watch where it actually goes, then draw the graph you observed. Designing the topology before you’ve seen one real run is how you get twelve nodes where four would do.

The shape outlives the framework

Every example here is LangGraph, because it’s what most people meet first. But nothing in the last four sections is LangGraph’s idea. Explicit states, allowed transitions, retries with a cap, a step that waits for a human — workflow engines were doing all of that a decade before anyone put a model inside a node. What’s new is only that some of the nodes now call one.

Choosing between them is a real decision, and “most explicit” isn’t automatically right — a handoff framework gives you less to configure, which is the better trade when the steps genuinely aren’t knowable. What ports is the three questions: what are the nodes, what is allowed to follow what, and where does a run park when something has to wait? Answer those and you can carry the design to whatever you use next quarter. Skip them and no framework saves you.

How to draw your first one

The fastest path from “I have an agent” to “I have a graph”, in the order that actually works:

• 1 · List the decisions, not the steps. Every place the flow can go two ways is an edge. Everything between two decisions is a node.
• 2 · Write the state first. Four or five fields. If you can’t name what moves between nodes, you don’t have the shape yet.
• 3 · Make every step you can deterministic. Lookups, validation, formatting — plain functions. Reserve model calls for judgement.
• 4 · Put a gate in front of anything irreversible. Money, email, deletes. A human checkpoint costs one node.
• 5 · Add a checkpointer on day one. Not when it breaks in production — you’ll want the run history the first time you debug a route.

If you’re preparing for an FDE or AI engineering interview, this is worth practising out loud: given a workflow, name the nodes, name the edges, say where the checkpoint goes and why. It’s the fastest way to show you’ve shipped one.