Function-calling and Tool-use

Tool-use (also called function-calling) is the protocol that lets an LLM ask your app to run a named function with a JSON argument, wait for the result, and keep reasoning. It is the mechanism behind every agent, copilot, and structured-output pipeline in production.

The loop

1. You define tools in JSON Schema and send them with the prompt.
2. The model responds with either a normal text message or a tool_use block: {"name": "get_weather", "input": {"city": "Tokyo"}}.
3. Your code executes the function and sends a tool_result back.
4. The model continues until it emits a final text answer.

All major APIs — OpenAI, Anthropic, Google — support the same shape with minor naming differences.

What tools should you expose?

• Database queries the agent can parameterize — NOT raw SQL execution.
• Narrow APIs with clear auth (get_order, refund_order, lookup_user).
• Local capabilities an agent framework orchestrates (shell, file write, search).

Keep descriptions concrete. search_users(query: string) with three examples beats a vague query_db(sql: string) — the model will abuse the vague one.

Failure modes

• Hallucinated arguments. The model invents fields not in your schema. Validate server-side; never trust the JSON blindly.
• Runaway loops. An agent calls search twelve times with tiny query variations. Cap tool-call depth.
• Parallel chaos. Parallel tool calls (supported by Claude, GPT) race when tools touch shared state. Serialize writes.
• Silent schema drift. You change an argument name; the model still sends the old name because your prompt shows old examples.

When NOT to use tools

If the answer lives in the prompt, don't bounce it through a tool. A single SELECT you could have inlined is cheaper, faster, and less likely to fail than a tool round-trip.