Tool calling

Tool calling lets a model ask your application to run a function, such as a database lookup or an API call, and then continue with the result. Numes uses the OpenAI Chat Completions tool format. The model never runs your functions: it returns a structured call, your code runs it, and you send the result back in the next request.

Numes checks tool_choice, parallel_tool_calls and max_tool_calls on every response, so these controls mean the same thing on every endpoint. For tools that run on the provider's side, such as web search, see Hosted tools.

How it works

1. 1

   Send tools with the request

   Describe each function with a name, a description and a JSON Schema for its arguments.

2. 2

   Run the calls the model returns

   If the model decides to call a tool, the response contains message.tool_calls. Parse each call's arguments and run the matching function in your code.

3. 3

   Send the results back

   Append the assistant message with its tool_calls, then one tool message per call with the matching tool_call_id. Send the conversation again; the model either answers or calls more tools.

   Tool-result text can include terminal output. JSON-escape control characters, such as \u001b for an ANSI escape; message content, reasoning state, tool and schema descriptions, and response text preserve those characters. Request, message and response size limits still apply. Tool-call and reasoning IDs are opaque strings of at most 256 UTF-8 bytes: return the exact ID with the result. Escape returned content when displaying it in HTML or a terminal.

   JSON numbers in tool schemas and replayed arguments preserve their precision through dispatch. Use a lossless JSON parser in your client when arguments can contain large integers or precise decimals; JavaScript's JSON.parse can round them before you send a continuation.

For streaming responses, collect delta.tool_calls by index and concatenate function.arguments fragments before parsing the JSON. Call IDs, names and argument bytes are sent once, including when a provider completes a streamed call.

Define tools

curl -X POST "https://api.numes.ai/api/v1/chat/completions" \  -H "Authorization: Bearer $NUMES_API_KEY" \  -H "Content-Type: application/json" \  -d '{    "model": "openai/gpt-6-luna",    "messages": [      {        "role": "user",        "content": "What is the weather in Lisbon right now?"      }    ],    "tools": [      {        "type": "function",        "function": {          "name": "get_weather",          "description": "Get the current weather for a city.",          "parameters": {            "type": "object",            "properties": {              "city": {                "type": "string",                "description": "City name, for example Lisbon"              },              "unit": {                "type": "string",                "enum": [                  "celsius",                  "fahrenheit"                ]              }            },            "required": [              "city"            ]          }        }      }    ]  }'

Function definitions are part of the prompt and count toward prompt tokens, so keep descriptions and schemas as short as the model needs.

Control tool use

Tool choice

tool_choice controls whether and which tool the model calls. "none" and a named function are guaranteed: a response that violates them is rejected rather than returned to you. "required" is passed to the provider but isn't guaranteed.

To guarantee a call, name the function. When a response is rejected before any output reaches you, Numes can retry on another eligible endpoint; otherwise the request fails with 502 provider_unavailable. A response cut off by max_tokens (finish_reason: "length") isn't rejected for a missing call.

Parallel calls

Models can return several calls in one response. Set parallel_tool_calls: false to allow at most one: a response that starts a second call is rejected. Omit the field to use the provider's default.

Call limit

max_tool_calls caps the number of distinct calls in one response, including hosted tool calls. It accepts 1 to 128; higher values are rejected with 400. A response that goes beyond it is rejected rather than silently truncated.

A complete round trip

1. The model requests a call

Sending the request from Define tools returns an assistant message with tool_calls instead of text:

{  "id": "0c8e4f7a-51d2-4b6e-9a3f-7d2c1e9b8a60",  "object": "chat.completion",  "created": 1790000000,  "model": "openai/gpt-6-luna",  "provider": "example-provider",  "choices": [    {      "index": 0,      "finish_reason": "tool_calls",      "message": {        "role": "assistant",        "content": null,        "tool_calls": [          {            "id": "call_7f3a9c",            "type": "function",            "function": {              "name": "get_weather",              "arguments": "{\"city\":\"Lisbon\",\"unit\":\"celsius\"}"            }          }        ]      }    }  ],  "usage": {    "prompt_tokens": 82,    "completion_tokens": 21,    "total_tokens": 103,    "cost": 0.000037,    "cost_details": { "upstream_inference_cost": 0.000037 }  }}

arguments is a JSON string. Parse it before use, and validate it: models can produce arguments that do not match your schema.

Some endpoints report finish_reason: "stop" even when the message contains tool calls. Decide on the presence of tool_calls, not on the finish reason alone.

2. Run the function

Your code runs get_weather(city="Lisbon", unit="celsius") and gets a result, for example {"temperature": 21, "conditions": "clear"}.

3. Send the result

Append the assistant message and a tool message whose tool_call_id matches the call's id, then send the request again with the same tools:

[  { "role": "user", "content": "What is the weather in Lisbon right now?" },  {    "role": "assistant",    "content": null,    "tool_calls": [      {        "id": "call_7f3a9c",        "type": "function",        "function": { "name": "get_weather", "arguments": "{\"city\":\"Lisbon\",\"unit\":\"celsius\"}" }      }    ]  },  {    "role": "tool",    "tool_call_id": "call_7f3a9c",    "content": "{\"temperature\":21,\"unit\":\"celsius\",\"conditions\":\"clear\"}"  }]

The model now answers with text, for example "It is 21 °C and clear in Lisbon."

Message format rules

Request messages preserve unfamiliar provider attributes. The upstream validates conversation semantics.

Agentic loop

This loop keeps calling the model until it answers without tools, with a turn limit so a confused model cannot loop forever. Errors from your functions go back to the model as results instead of aborting the run.

import jsonimport osfrom openai import OpenAI​client = OpenAI(base_url="https://api.numes.ai/api/v1", api_key=os.environ["NUMES_API_KEY"])​​def get_weather(city: str, unit: str = "celsius") -> dict:    return {"city": city, "temperature": 21, "unit": unit, "conditions": "clear"}​​HANDLERS = {"get_weather": get_weather}TOOLS = [{    "type": "function",    "function": {        "name": "get_weather",        "description": "Get the current weather for a city.",        "parameters": {            "type": "object",            "properties": {                "city": {"type": "string"},                "unit": {"type": "string", "enum": ["celsius", "fahrenheit"]},            },            "required": ["city"],        },    },}]​​def run_tool(name: str, arguments: str) -> str:    handler = HANDLERS.get(name)    if handler is None:        return json.dumps({"error": f"unknown tool {name}"})    try:        return json.dumps(handler(**json.loads(arguments)))    except Exception as exc:  # report the failure to the model        return json.dumps({"error": str(exc)})​​def run(messages: list[dict], max_turns: int = 8) -> str:    for _ in range(max_turns):        response = client.chat.completions.create(            model="openai/gpt-6-luna",            messages=messages,            tools=TOOLS,        )        message = response.choices[0].message        if not message.tool_calls:            return message.content or ""​        assistant = {            "role": "assistant",            "content": message.content,            "tool_calls": [                {                    "id": call.id,                    "type": "function",                    "function": {"name": call.function.name, "arguments": call.function.arguments},                }                for call in message.tool_calls            ],        }        details = getattr(message, "reasoning_details", None)        if details:            assistant["reasoning_details"] = details  # return reasoning state unchanged        messages.append(assistant)​        for call in message.tool_calls:            messages.append({                "role": "tool",                "tool_call_id": call.id,                "content": run_tool(call.function.name, call.function.arguments),            })    raise RuntimeError("the model did not finish within the turn limit")​​print(run([{"role": "user", "content": "Compare the weather in Lisbon and Porto."}]))

When the model returns several calls in one response, the calls are independent: you can run them concurrently, but append the tool messages in the same order as the calls.

Stream tool calls

With stream: true, calls arrive in delta.tool_calls, keyed by index. The delta that opens a call carries id, type and function.name; function.arguments arrives whole or in fragments. Accumulate by index and run tools after the finish chunk.

import json​​def complete(arguments: str) -> bool:    try:        return isinstance(json.loads(arguments), dict)    except ValueError:        return False​​stream = client.chat.completions.create(    model="openai/gpt-6-luna",    messages=messages,    tools=TOOLS,    stream=True,)​calls: dict[int, dict] = {}content = []for chunk in stream:    if not chunk.choices:        continue  # the usage-only chunk    delta = chunk.choices[0].delta    if delta.content:        content.append(delta.content)    for part in delta.tool_calls or []:        call = calls.setdefault(part.index, {"id": "", "name": "", "arguments": ""})        if part.id:            call["id"] = part.id        if part.function and part.function.name:            call["name"] = part.function.name        if part.function and part.function.arguments and not complete(call["arguments"]):            call["arguments"] += part.function.arguments​tool_calls = [calls[index] for index in sorted(calls)]

The complete check stops appending once a call's arguments form a JSON object, which keeps the result valid if an endpoint repeats the full arguments at the end of a call. Streaming describes the event format and error handling.

Continue with reasoning

Reasoning models can return reasoning_details alongside tool calls. For some providers these carry signed or encrypted state that the model needs to continue after the tool results. Send them back unchanged on the assistant message that carries the calls, as the loop above does. When streaming, accumulate the fragments in order first. Unfamiliar detail types and fields are preserved. See Reasoning.

Endpoint support

Client functions are forwarded by default. With provider.require_parameters: true, active tools require declared tool support; if none is eligible, the request fails with 503 provider_unavailable. Hosted tools always retain capability and execution checks. Filter the catalogue for tool-capable models:

curl "https://api.numes.ai/api/v1/models?supported_parameters=tools" \  -H "Authorization: Bearer $NUMES_API_KEY"

Endpoints differ in which tool controls they can guarantee. The supported_parameters of each endpoint lists tools, and also tool_choice and parallel_tool_calls where the endpoint supports them:

curl "https://api.numes.ai/api/v1/models/openai/gpt-6-luna/endpoints" \  -H "Authorization: Bearer $NUMES_API_KEY"

Some endpoints support only automatic tool calls. With provider.require_parameters: true, a tool_choice of "required" or a named function routes only to endpoints that list tool_choice; "auto" and "none" never narrow routing. parallel_tool_calls isn't covered by that flag, so pin endpoints that list it with provider.only or provider.order when you depend on it. See Provider routing.

Usage and billing

• Function definitions sent to the provider count as prompt tokens. With tool_choice: "none" no definitions are sent; with a named choice only that function is sent.

• Tool calls count as completion tokens, together with text and reasoning. max_tokens caps them combined.

• The tool results you send back are ordinary input on the next request.

See Usage accounting for how usage and cost are reported.

Errors

Best practices

• Name functions portably. Use letters, digits, underscores and hyphens; some providers reject other characters.

• Write descriptions for the model. State what the function does, when to use it and what each argument means.

• Validate arguments before executing. Treat them as untrusted input, especially for functions with side effects.

• Return errors as results. A result such as {"error": "city not found"} lets the model recover.

• Bound the loop. Cap the number of turns and set max_tool_calls to what one step needs.

• Send the same tools on every turn so the model can call again after reading results.

• Keep a conversation on one endpoint. Signed reasoning state belongs to the provider that produced it; a session ID keeps follow-up turns on the same endpoint when it is still eligible.

Next steps