How LLMs Call Tools
How LLMs use function calling and tool use — the mechanics behind tool-calling agents, from prompt engineering to structured output.
Last updated
After this section you can
- Describe the runtime loop precisely: the model emits structured text, your code executes it, the result comes back as a message
- Explain what the model actually sees of a tool, and why the description does most of the work
- Tell sequential from parallel tool calls and predict the latency difference
- Name the common tool-calling failure modes and the mitigation for each
02
How LLMs Actually Call Tools
LLMs never execute code — they emit structured text that your system runs. Master this one mechanism and every agent, from a support bot to a coding assistant, stops being magic and starts being architecture.
KEY INSIGHT
Tool calling is an illusion of action. The model outputs a JSON object that looks like a function call. Your code parses it, runs the real function, and feeds the result back as more text. The model never touches your APIs, databases, or file system — it only ever reads and writes tokens.
The Tool-Calling Loop — who does what