Ask a chat model to check the weather and it will happily invent a forecast — it has no way to reach the outside world. Function calling, also called tool calling, is the mechanism that changes this: you describe functions the model may use, the model responds with a structured request to invoke one, your code executes it, and the result flows back into the conversation. Every agent framework you have seen is a thin layer over this loop. Understanding what the API actually does — and where it silently fails — is the difference between a demo and a production agent.
This post walks through the loop end to end using the two API families that most implementations converge on: Anthropic’s tool use API and OpenAI’s function calling. The concepts transfer to almost every other provider, because they all copied the same shape.
A Tool Is a Description, Not a Function
The first thing to internalize is that you never send code to the model. You send a JSON description — a name, a description, and a JSON Schema for the arguments. The model’s job is to output a structured object claiming which tool to call and with what arguments; nothing executes until your code says so. Here is a tool definition for the Anthropic Messages API:
tools = [
{
"name": "get_weather",
"description": (
"Get the current weather for a city. Returns temperature, "
"conditions, and a short forecast. Use for any question about "
"current or upcoming weather. Does not cover historical data."
),
"input_schema": {
"type": "object",
"properties": {
"city": {
"type": "string",
"description": "City name, e.g. 'Oslo, Norway'",
},
"unit": {
"type": "string",
"enum": ["celsius", "fahrenheit"],
"description": "Temperature unit, defaults to celsius",
},
},
"required": ["city"],
},
}
]
The description is doing far more work than it looks like. Under the hood, the provider turns your tool definitions into part of the prompt — Anthropic’s API, for example, constructs a system prompt that enumerates the available functions in JSON Schema form before your own system prompt. The model chooses tools based on that text, so a vague description produces vague tool selection. The provider guidance is blunt and matches what practitioners see: write several sentences per tool covering what it does, when to use it, when not to use it, and what each parameter means. Two more guidelines that measurably reduce misfires as your tool count grows: consolidate related operations into one tool with an action parameter rather than five near-identical tools, and namespace names by service (github_list_prs, not list_prs) so selection stays unambiguous.
The Loop: Tool Calls In, Tool Results Back
Function calling is never a single request. It is a loop your application drives: send the conversation, receive tool calls, execute them, append the results, and send everything back. The model either emits more tool calls or produces a final text answer. Here is the complete loop against the Anthropic API:
import json
import anthropic
client = anthropic.Anthropic()
MODEL = "claude-opus-5-5"
def run_agent(user_msg: str, tools: list) -> str:
messages = [{"role": "user", "content": user_msg}]
while True:
resp = client.messages.create(
model=MODEL,
max_tokens=1024,
tools=tools,
messages=messages,
)
messages.append({"role": "assistant", "content": resp.content})
calls = [b for b in resp.content if b.type == "tool_use"]
if not calls:
# No tool_use blocks: the model answered in plain text.
return "".join(b.text for b in resp.content if b.type == "text")
results = []
for call in calls:
output = dispatch(call.name, call.input) # your tool registry
results.append({
"type": "tool_result",
"tool_use_id": call.id,
"content": json.dumps(output),
})
messages.append({"role": "user", "content": results})
Three structural details trip up first implementations. The assistant’s turn — including the tool_use blocks themselves — must be appended to the history exactly as received, because the results reference the call IDs. Tool results are not a special endpoint; they are tool_result content blocks inside the next user message. And the loop needs a termination condition you control: a maximum iteration count, so a model that keeps calling tools in circles cannot run up your bill forever.
The dispatch function is the boundary between model output and your system, and it deserves the same skepticism you would apply to any untrusted input:
def dispatch(name: str, args: dict) -> dict:
if name == "get_weather":
city = args.get("city")
if not isinstance(city, str) or not city.strip():
return {"error": "'city' must be a non-empty string"}
return weather_lookup(city.strip(), unit=args.get("unit", "celsius"))
return {"error": f"unknown tool '{name}'. Available: get_weather"}
Validation failures should go back to the model as a tool result, not raise an exception. A well-formed error message lets the model correct its arguments and retry; an unhandled exception kills the loop. This is also why unknown tool names get a listing of what actually exists — occasionally the model hallucinates a tool name, and the correction steers it back.
Strict Schemas and Grammar Constraints
OpenAI’s variant of the same mechanism adds a mode worth knowing about. Function tools look similar — a name, description, JSON Schema parameters — but with strict: true the provider guarantees the generated arguments conform to the schema, using constrained decoding under the hood. The trade-off is that strict mode restricts the schema: every property must be listed in required and additionalProperties must be false, so “optional” fields are expressed as nullable types instead.
openai_tools = [
{
"type": "function",
"function": {
"name": "get_weather",
"description": "Get the current weather for a city.",
"parameters": {
"type": "object",
"properties": {
"city": {"type": "string"},
"unit": {"type": "string", "enum": ["celsius", "fahrenheit"]},
},
"required": ["city", "unit"],
"additionalProperties": False,
},
"strict": True,
},
}
]
Handling the calls differs slightly from Anthropic’s shape: the assistant message carries a tool_calls list where each entry’s function.arguments is a JSON string you parse yourself, and results go back as messages with role: "tool" and the matching tool_call_id:
import json
import os
resp = client.chat.completions.create(
model=os.environ["OPENAI_MODEL"], tools=openai_tools, messages=messages
)
msg = resp.choices[0].message
if msg.tool_calls:
followup = []
for call in msg.tool_calls:
args = json.loads(call.function.arguments)
followup.append({
"role": "tool",
"tool_call_id": call.id,
"content": json.dumps(dispatch(call.function.name, args)),
})
messages.append(msg) # assistant turn, tool calls included
messages.extend(followup) # one role="tool" message per call
Two OpenAI-specific notes for 2026-era integrations. First, the newest flagship models (GPT-6 Astra and GPT-6.1 Sol) require the Responses API rather than Chat Completions for tool calling, so check which surface your model family supports before standardizing on one request shape. Second, beyond schema-strict function tools, the Responses API also offers custom tools that take free-form text or regex/grammar-constrained input — useful for arguments like dates or code fragments where JSON is awkward. Note that regex constraints use the Rust regex crate syntax, so lookarounds and lazy modifiers are not available. And if your application exposes dozens or hundreds of tools, tool search lets the model load definitions on demand instead of stuffing every schema into the context window.
tool_choice: Who Decides
By default the model decides whether to call a tool, which is what you want for chat-style agents. Both APIs let you override this. Anthropic’s tool_choice accepts {"type": "auto"}, {"type": "any"} (must call something, its choice), {"type": "tool", "name": "..."} (must call this specific tool), and {"type": "none"}. Forcing a call is useful for structured extraction — point the model at a single tool whose input schema is your target output shape, and the “function call” becomes a typed extraction result. One behavioral caveat: forcing a tool use skips extended thinking on models that support it, so if you need the model to reason before acting, use auto with an instruction in the prompt instead.
What Production Agents Get Wrong
- Trusting the arguments. Model-generated arguments are untrusted input. Validate types and ranges before execution, and never let a tool argument reach a SQL query, filesystem path, or shell command unescaped.
- Unbounded side effects. A tool that charges a card or deletes a row needs an idempotency key and a timeout. Models occasionally re-issue a call with identical arguments; your tools should tolerate that.
- Prompt injection through results. Tool output becomes context the model reasons over. A page fetched by your web-browsing tool can contain text that says “ignore your instructions.” Treat tool results the way you treat user input, and keep privileged operations behind explicit confirmation.
- Bloated results. Returning a 50KB JSON dump wastes context and degrades the model’s next decision. Return the fields the model needs, with stable identifiers it can reference in follow-up calls.
Once you have more than a handful of tools, it is worth looking at the Model Context Protocol, which standardizes how tool definitions, invocations, and results are exchanged between a host application and tool servers — the same loop described here, with a wire format so tools can be shared across clients.
The core loop has barely changed since function calling shipped: describe tools, receive calls, execute, return results. The provider features layered on top — strict schemas, grammar constraints, tool search, choice forcing — all tune one of those four steps. Get the loop’s boundaries right, and the tuning is straightforward.