# Tool calling

> Let models call functions that your code runs, and control which tools they call, how many and in what form.

Source: https://0.0.0.0:3000/docs/features/tool-calling
Documentation index: https://0.0.0.0:3000/docs/llms.txt

Tool calling lets a model ask your application to run a function, such as a database lookup or an API call, and then continue with the result. Numes uses the OpenAI Chat Completions tool format. The model never runs your functions: it returns a structured call, your code runs it, and you send the result back in the next request.

Numes checks `tool_choice`, `parallel_tool_calls` and `max_tool_calls` on every response, so these controls mean the same thing on every endpoint. For tools that run on the provider's side, such as web search, see [Hosted tools](/docs/features/hosted-tools).

## How it works

:::steps
### Send tools with the request

Describe each function with a name, a description and a JSON Schema for its arguments.

### Run the calls the model returns

If the model decides to call a tool, the response contains `message.tool_calls`. Parse each call's `arguments` and run the matching function in your code.

### Send the results back

Append the assistant message with its `tool_calls`, then one `tool` message per call with the matching `tool_call_id`. Send the conversation again; the model either answers or calls more tools.

Tool-result text can include terminal output. JSON-escape control characters,
such as `\u001b` for an ANSI escape; message content, reasoning state, tool
and schema descriptions, and response text preserve those characters. Request,
message and response size limits still apply. Tool-call and reasoning IDs are
opaque strings of at most 256 UTF-8 bytes: return the exact ID with the result.
Escape returned content when displaying it in HTML or a terminal.

JSON numbers in tool schemas and replayed arguments preserve their precision
through dispatch. Use a lossless JSON parser in your client when arguments can
contain large integers or precise decimals; JavaScript's `JSON.parse` can round
them before you send a continuation.
:::

For streaming responses, collect `delta.tool_calls` by `index` and concatenate
`function.arguments` fragments before parsing the JSON. Call IDs, names and
argument bytes are sent once, including when a provider completes a streamed call.

## Define tools

```request
{
  "method": "POST",
  "path": "/chat/completions",
  "body": {
    "model": "openai/gpt-6-luna",
    "messages": [
      { "role": "user", "content": "What is the weather in Lisbon right now?" }
    ],
    "tools": [
      {
        "type": "function",
        "function": {
          "name": "get_weather",
          "description": "Get the current weather for a city.",
          "parameters": {
            "type": "object",
            "properties": {
              "city": { "type": "string", "description": "City name, for example Lisbon" },
              "unit": { "type": "string", "enum": ["celsius", "fahrenheit"] }
            },
            "required": ["city"]
          }
        }
      }
    ]
  }
}
```

:::fields Function tool
- `type` `string` required — `function`.
- `function` `object` required — The function definition.
  - `name` `string` required — Provider-owned function name. The upstream validates names and duplicates.
  - `description` `string` — What the function does and when to use it.
  - `parameters` `object` required — JSON Schema object describing the arguments. Use `{"type": "object", "properties": {}}` for a function without arguments.
  - `strict` `boolean` — Asks the provider to enforce the schema exactly. Forwarded to endpoints that support it; see [Endpoint support](#endpoint-support).
:::

| Limit | Value |
| --- | --- |
| Tools per request (function and hosted combined) | No limit; the request body limit applies |
| Function name | Validated by the upstream |
| Function description | No limit |
| Parameter schema | JSON object, no size limit |
| Calls per response (`max_tool_calls`) | 1–128, default 128 |
| `tool_calls` per assistant message in history | 256 |

Function definitions are part of the prompt and count toward prompt tokens, so keep descriptions and schemas as short as the model needs.

## Control tool use

### Tool choice

`tool_choice` controls whether and which tool the model calls. `"none"` and a named function are guaranteed: a response that violates them is rejected rather than returned to you. `"required"` is passed to the provider but isn't guaranteed.

| Value | Behavior |
| --- | --- |
| `"auto"` (default) | The model decides whether to call tools. |
| `"none"` | Tools aren't sent to the provider, and a tool call in the response is rejected. Tools then don't narrow routing. |
| `"required"` | The provider is asked to call at least one tool. Most do, but a response without a tool call is returned and billed like any other. Requires a non-empty `tools` array. |
| `{"type": "function", "function": {"name": "get_weather"}}` | Only the named function is sent to the provider, and the response must call it. The name must match a declared function. |

To guarantee a call, name the function. When a response is rejected before any output reaches you, Numes can retry on another eligible endpoint; otherwise the request fails with `502 provider_unavailable`. A response cut off by `max_tokens` (`finish_reason: "length"`) isn't rejected for a missing call.

### Parallel calls

Models can return several calls in one response. Set `parallel_tool_calls: false` to allow at most one: a response that starts a second call is rejected. Omit the field to use the provider's default.

### Call limit

`max_tool_calls` caps the number of distinct calls in one response, including [hosted tool](/docs/features/hosted-tools) calls. It accepts 1 to 128; higher values are rejected with `400`. A response that goes beyond it is rejected rather than silently truncated.

## A complete round trip

### 1. The model requests a call

Sending the request from [Define tools](#define-tools) returns an assistant message with `tool_calls` instead of text:

```json title="Response"
{
  "id": "0c8e4f7a-51d2-4b6e-9a3f-7d2c1e9b8a60",
  "object": "chat.completion",
  "created": 1790000000,
  "model": "openai/gpt-6-luna",
  "provider": "example-provider",
  "choices": [
    {
      "index": 0,
      "finish_reason": "tool_calls",
      "message": {
        "role": "assistant",
        "content": null,
        "tool_calls": [
          {
            "id": "call_7f3a9c",
            "type": "function",
            "function": {
              "name": "get_weather",
              "arguments": "{\"city\":\"Lisbon\",\"unit\":\"celsius\"}"
            }
          }
        ]
      }
    }
  ],
  "usage": {
    "prompt_tokens": 82,
    "completion_tokens": 21,
    "total_tokens": 103,
    "cost": 0.000037,
    "cost_details": { "upstream_inference_cost": 0.000037 }
  }
}
```

`arguments` is a JSON **string**. Parse it before use, and validate it: models can produce arguments that do not match your schema.

Some endpoints report `finish_reason: "stop"` even when the message contains tool calls. Decide on the presence of `tool_calls`, not on the finish reason alone.

### 2. Run the function

Your code runs `get_weather(city="Lisbon", unit="celsius")` and gets a result, for example `{"temperature": 21, "conditions": "clear"}`.

### 3. Send the result

Append the assistant message and a `tool` message whose `tool_call_id` matches the call's `id`, then send the request again with the same `tools`:

```json title="messages in the next request"
[
  { "role": "user", "content": "What is the weather in Lisbon right now?" },
  {
    "role": "assistant",
    "content": null,
    "tool_calls": [
      {
        "id": "call_7f3a9c",
        "type": "function",
        "function": { "name": "get_weather", "arguments": "{\"city\":\"Lisbon\",\"unit\":\"celsius\"}" }
      }
    ]
  },
  {
    "role": "tool",
    "tool_call_id": "call_7f3a9c",
    "content": "{\"temperature\":21,\"unit\":\"celsius\",\"conditions\":\"clear\"}"
  }
]
```

The model now answers with text, for example "It is 21 °C and clear in Lisbon."

## Message format rules

Request messages preserve unfamiliar provider attributes. The upstream validates conversation semantics.

:::fields Assistant message with tool calls
- `role` `string` required — `assistant`.
- `content` `string | null` — May be `null` or empty when the message carries tool calls.
- `tool_calls` `object[]` — Historical provider calls.
  - `id` `string` required — The call ID from the response.
  - `type` `string` required — `function`.
  - `function` `object` required — `name` and an `arguments` string, usually JSON such as `"{}"`. Historical argument strings are preserved unchanged.
- `reasoning_details` `object[]` — Reasoning state returned with the calls. See [Continue with reasoning](#continue-with-reasoning).
:::

:::fields Tool result message
- `role` `string` required — `tool`.
- `tool_call_id` `string` required — The `id` of the call this result answers.
- `content` `string | object[]` required — The result, usually a JSON string. Empty results are accepted. The upstream validates message semantics; media retains acquisition and accounting controls.
- `name` `string` — Provider message metadata, forwarded by matching compatible adapters.
:::

:::warning Build the assistant message explicitly
Preserve `content`, `tool_calls`, `reasoning_details` and provider continuation attributes when appending an assistant response. Matching compatible endpoints accept unfamiliar message fields; native adapters may reject fields they cannot translate.
:::

## Agentic loop

This loop keeps calling the model until it answers without tools, with a turn limit so a confused model cannot loop forever. Errors from your functions go back to the model as results instead of aborting the run.

:::code-group
```python title="Python"
import json
import os
from openai import OpenAI

client = OpenAI(base_url="https://api.numes.ai/api/v1", api_key=os.environ["NUMES_API_KEY"])


def get_weather(city: str, unit: str = "celsius") -> dict:
    return {"city": city, "temperature": 21, "unit": unit, "conditions": "clear"}


HANDLERS = {"get_weather": get_weather}
TOOLS = [{
    "type": "function",
    "function": {
        "name": "get_weather",
        "description": "Get the current weather for a city.",
        "parameters": {
            "type": "object",
            "properties": {
                "city": {"type": "string"},
                "unit": {"type": "string", "enum": ["celsius", "fahrenheit"]},
            },
            "required": ["city"],
        },
    },
}]


def run_tool(name: str, arguments: str) -> str:
    handler = HANDLERS.get(name)
    if handler is None:
        return json.dumps({"error": f"unknown tool {name}"})
    try:
        return json.dumps(handler(**json.loads(arguments)))
    except Exception as exc:  # report the failure to the model
        return json.dumps({"error": str(exc)})


def run(messages: list[dict], max_turns: int = 8) -> str:
    for _ in range(max_turns):
        response = client.chat.completions.create(
            model="openai/gpt-6-luna",
            messages=messages,
            tools=TOOLS,
        )
        message = response.choices[0].message
        if not message.tool_calls:
            return message.content or ""

        assistant = {
            "role": "assistant",
            "content": message.content,
            "tool_calls": [
                {
                    "id": call.id,
                    "type": "function",
                    "function": {"name": call.function.name, "arguments": call.function.arguments},
                }
                for call in message.tool_calls
            ],
        }
        details = getattr(message, "reasoning_details", None)
        if details:
            assistant["reasoning_details"] = details  # return reasoning state unchanged
        messages.append(assistant)

        for call in message.tool_calls:
            messages.append({
                "role": "tool",
                "tool_call_id": call.id,
                "content": run_tool(call.function.name, call.function.arguments),
            })
    raise RuntimeError("the model did not finish within the turn limit")


print(run([{"role": "user", "content": "Compare the weather in Lisbon and Porto."}]))
```

```typescript title="TypeScript"
import OpenAI from "openai";
import type {
  ChatCompletionMessageParam,
  ChatCompletionTool,
} from "openai/resources/chat/completions";

const client = new OpenAI({ baseURL: "https://api.numes.ai/api/v1", apiKey: process.env.NUMES_API_KEY });

const handlers: Record<string, (args: any) => unknown> = {
  get_weather: ({ city, unit = "celsius" }: { city: string; unit?: string }) => ({
    city,
    temperature: 21,
    unit,
    conditions: "clear",
  }),
};

const tools: ChatCompletionTool[] = [
  {
    type: "function",
    function: {
      name: "get_weather",
      description: "Get the current weather for a city.",
      parameters: {
        type: "object",
        properties: {
          city: { type: "string" },
          unit: { type: "string", enum: ["celsius", "fahrenheit"] },
        },
        required: ["city"],
      },
    },
  },
];

function runTool(name: string, args: string): string {
  const handler = handlers[name];
  if (!handler) return JSON.stringify({ error: `unknown tool ${name}` });
  try {
    return JSON.stringify(handler(JSON.parse(args)));
  } catch (error) {
    return JSON.stringify({ error: String(error) }); // report the failure to the model
  }
}

async function run(messages: ChatCompletionMessageParam[], maxTurns = 8): Promise<string> {
  for (let turn = 0; turn < maxTurns; turn++) {
    const response = await client.chat.completions.create({
      model: "openai/gpt-6-luna",
      messages,
      tools,
    });
    const message = response.choices[0].message;
    const calls: { id: string; name: string; arguments: string }[] = [];
    for (const call of message.tool_calls ?? []) {
      if (call.type === "function") {
        calls.push({ id: call.id, name: call.function.name, arguments: call.function.arguments });
      }
    }
    if (calls.length === 0) return message.content ?? "";

    const details = (message as { reasoning_details?: unknown[] }).reasoning_details;
    messages.push({
      role: "assistant",
      content: message.content,
      tool_calls: calls.map((call) => ({
        id: call.id,
        type: "function" as const,
        function: { name: call.name, arguments: call.arguments },
      })),
      ...(details?.length ? { reasoning_details: details } : {}), // return reasoning state unchanged
    } as ChatCompletionMessageParam);

    for (const call of calls) {
      messages.push({ role: "tool", tool_call_id: call.id, content: runTool(call.name, call.arguments) });
    }
  }
  throw new Error("the model did not finish within the turn limit");
}

console.log(await run([{ role: "user", content: "Compare the weather in Lisbon and Porto." }]));
```
:::

When the model returns several calls in one response, the calls are independent: you can run them concurrently, but append the `tool` messages in the same order as the calls.

## Stream tool calls

With `stream: true`, calls arrive in `delta.tool_calls`, keyed by `index`. The delta that opens a call carries `id`, `type` and `function.name`; `function.arguments` arrives whole or in fragments. Accumulate by `index` and run tools after the finish chunk.

:::code-group
```python title="Python"
import json


def complete(arguments: str) -> bool:
    try:
        return isinstance(json.loads(arguments), dict)
    except ValueError:
        return False


stream = client.chat.completions.create(
    model="openai/gpt-6-luna",
    messages=messages,
    tools=TOOLS,
    stream=True,
)

calls: dict[int, dict] = {}
content = []
for chunk in stream:
    if not chunk.choices:
        continue  # the usage-only chunk
    delta = chunk.choices[0].delta
    if delta.content:
        content.append(delta.content)
    for part in delta.tool_calls or []:
        call = calls.setdefault(part.index, {"id": "", "name": "", "arguments": ""})
        if part.id:
            call["id"] = part.id
        if part.function and part.function.name:
            call["name"] = part.function.name
        if part.function and part.function.arguments and not complete(call["arguments"]):
            call["arguments"] += part.function.arguments

tool_calls = [calls[index] for index in sorted(calls)]
```

```typescript title="TypeScript"
type PendingCall = { id: string; name: string; arguments: string };

const complete = (args: string): boolean => {
  try {
    const value = JSON.parse(args);
    return typeof value === "object" && value !== null && !Array.isArray(value);
  } catch {
    return false;
  }
};

const stream = await client.chat.completions.create({
  model: "openai/gpt-6-luna",
  messages,
  tools,
  stream: true,
});

const calls = new Map<number, PendingCall>();
let content = "";
for await (const chunk of stream) {
  const delta = chunk.choices[0]?.delta; // undefined on the usage-only chunk
  if (!delta) continue;
  if (delta.content) content += delta.content;
  for (const part of delta.tool_calls ?? []) {
    const call = calls.get(part.index) ?? { id: "", name: "", arguments: "" };
    if (part.id) call.id = part.id;
    if (part.function?.name) call.name = part.function.name;
    if (part.function?.arguments && !complete(call.arguments)) call.arguments += part.function.arguments;
    calls.set(part.index, call);
  }
}

const toolCalls = [...calls.entries()].sort(([a], [b]) => a - b).map(([, call]) => call);
```
:::

The `complete` check stops appending once a call's arguments form a JSON object, which keeps the result valid if an endpoint repeats the full arguments at the end of a call. [Streaming](/docs/features/streaming) describes the event format and error handling.

## Continue with reasoning

Reasoning models can return `reasoning_details` alongside tool calls. For some providers these carry signed or encrypted state that the model needs to continue after the tool results. Send them back **unchanged** on the assistant message that carries the calls, as the loop above does. When streaming, accumulate the fragments in order first. Unfamiliar detail types and fields are preserved. See [Reasoning](/docs/features/reasoning#preserve-reasoning-across-turns).

## Endpoint support

Client functions are forwarded by default. With `provider.require_parameters: true`, active tools require declared tool support; if none is eligible, the request fails with `503 provider_unavailable`. Hosted tools always retain capability and execution checks. Filter the catalogue for tool-capable models:

```request
{
  "method": "GET",
  "path": "/models",
  "query": { "supported_parameters": "tools" }
}
```

Endpoints differ in which tool controls they can guarantee. The `supported_parameters` of each endpoint lists `tools`, and also `tool_choice` and `parallel_tool_calls` where the endpoint supports them:

```request
{
  "method": "GET",
  "path": "/models/openai/gpt-6-luna/endpoints"
}
```

Some endpoints support only automatic tool calls. With `provider.require_parameters: true`, a `tool_choice` of `"required"` or a named function routes only to endpoints that list `tool_choice`; `"auto"` and `"none"` never narrow routing. `parallel_tool_calls` isn't covered by that flag, so pin endpoints that list it with `provider.only` or `provider.order` when you depend on it. See [Provider routing](/docs/routing/provider-routing#require-parameters).

## Usage and billing

- Function definitions sent to the provider count as prompt tokens. With `tool_choice: "none"` no definitions are sent; with a named choice only that function is sent.
- Tool calls count as completion tokens, together with text and reasoning. `max_tokens` caps them combined.
- The tool results you send back are ordinary input on the next request.

See [Usage accounting](/docs/account/usage-accounting) for how usage and cost are reported.

## Errors

| Status | `error_type` | Cause |
| --- | --- | --- |
| `400` | `invalid_request` | Invalid owned execution control, `"required"` without tools, or `max_tool_calls` above the cap. The upstream validates function schemas and conversation semantics. |
| `502` | `provider_unavailable` | The provider's output violated `tool_choice`, `parallel_tool_calls` or `max_tool_calls`, or contained an invalid call, and no other endpoint succeeded. |
| `503` | `provider_unavailable` | No eligible endpoint supports tools for this model. |

## Best practices

- **Name functions portably.** Use letters, digits, underscores and hyphens; some providers reject other characters.
- **Write descriptions for the model.** State what the function does, when to use it and what each argument means.
- **Validate arguments before executing.** Treat them as untrusted input, especially for functions with side effects.
- **Return errors as results.** A result such as `{"error": "city not found"}` lets the model recover.
- **Bound the loop.** Cap the number of turns and set `max_tool_calls` to what one step needs.
- **Send the same `tools` on every turn** so the model can call again after reading results.
- **Keep a conversation on one endpoint.** Signed reasoning state belongs to the provider that produced it; a [session ID](/docs/features/sessions) keeps follow-up turns on the same endpoint when it is still eligible.

## Next steps

:::cards
- {globe} [Hosted tools](/docs/features/hosted-tools) Web search, code execution and other tools that run upstream.
- {waves} [Streaming](/docs/features/streaming) Event format, finish chunk and in-band errors.
- {braces} [Structured outputs](/docs/features/structured-outputs) Constrain the final answer to a JSON schema.
- {brain} [Reasoning](/docs/features/reasoning) Keep reasoning state intact through tool loops.
- {network} [Provider routing](/docs/routing/provider-routing) Pin, order and filter endpoints.
:::