Features

Structured outputs

Get answers as a JSON object or as JSON that follows your schema with response_format, and validate the result.

On this page

Choose a format
Request a JSON schema
JSON object check
Routing and support
Validate on the client
Stream structured output
Structured output with tools
Errors
Tips
Next steps

Structured outputs make a model answer with machine-readable JSON instead of prose. Set response_format on a chat completion to request a JSON object or an answer that follows your JSON Schema. The format is forwarded by matching compatible adapters. Native adapters translate supported formats to the source dialect. Set provider.require_parameters: true to require declared support.

How closely the answer follows your schema depends on the model and endpoint, so validate the parsed result in your application.

Choose a format

`response_format`	Output	Routing requirement
`{"type": "text"}`	Plain text. Same as omitting the field.	None
`{"type": "json_object"}`	Any JSON object.	Endpoint supports `response_format`
`{"type": "json_schema", "json_schema": {...}}`	A JSON object following your schema.	Endpoint supports `structured_outputs`

Prefer json_schema whenever you know the shape you need: the schema reaches the provider's native control, which constrains generation far more reliably than instructions alone.

response_format

typestringRequired

text, json_object or json_schema.

json_schemaobject

Required when type is json_schema, and not allowed otherwise.

Show child attributes

namestringRequired: Provider schema identifier.
descriptionstring: What the output represents.
schemaobjectRequired: The JSON Schema. Its root should describe an object.
strictboolean: Asks the provider to follow the schema exactly, where the provider supports strict mode.

Matching compatible adapters forward unfamiliar format and schema fields. The upstream validates their semantics.

Request a JSON schema

curl -X POST "https://api.numes.ai/api/v1/chat/completions" \  -H "Authorization: Bearer $NUMES_API_KEY" \  -H "Content-Type: application/json" \  -d '{    "model": "openai/gpt-6-luna",    "messages": [      {        "role": "system",        "content": "Extract the calendar event from the user'\''s message."      },      {        "role": "user",        "content": "Design review with Ana and Tomás next Tuesday, 3 March 2026, in room 4B."      }    ],    "response_format": {      "type": "json_schema",      "json_schema": {        "name": "calendar_event",        "description": "A single calendar event extracted from text.",        "strict": true,        "schema": {          "type": "object",          "properties": {            "title": {              "type": "string"            },            "date": {              "type": "string",              "description": "ISO 8601 date, YYYY-MM-DD"            },            "participants": {              "type": "array",              "items": {                "type": "string"              }            },            "location": {              "type": [                "string",                "null"              ]            }          },          "required": [            "title",            "date",            "participants",            "location"          ],          "additionalProperties": false        }      }    }  }'

The answer arrives as a JSON string in message.content:

Response excerpt

{  "choices": [    {      "index": 0,      "finish_reason": "stop",      "message": {        "role": "assistant",        "content": "{\"title\":\"Design review\",\"date\":\"2026-03-03\",\"participants\":[\"Ana\",\"Tomás\"],\"location\":\"Room 4B\"}"      }    }  ]}

For json_object, send {"type": "json_object"} and describe the fields you expect in the prompt. Some providers require the word "JSON" to appear in the messages for this mode, so mention it explicitly.

JSON object check

When type is json_object or json_schema, the completed answer must parse as a JSON object. A top-level array, a bare value, text around the JSON or Markdown code fences all fail. Output is never repaired or rewritten. The check doesn't apply to responses with tool calls or to answers cut off at max_tokens (finish_reason: "length"), which are usually incomplete JSON.

When a buffered answer fails the check, Numes can retry on another eligible endpoint; if none succeeds, you receive 502 provider_unavailable. On a stream the text has already reached you, so the stream ends with an in-band error chunk and the delivered output is billed. See Streaming.

Routing and support

With provider.require_parameters: true, structured output narrows routing to declared support. Without it, the selected upstream accepts or rejects the format.

Find models whose representative offer supports JSON schemas:

curl "https://api.numes.ai/api/v1/models?supported_parameters=structured_outputs" \  -H "Authorization: Bearer $NUMES_API_KEY"

Support can differ between endpoints of the same model. Each endpoint lists structured_outputs and response_format in its supported_parameters:

curl "https://api.numes.ai/api/v1/models/openai/gpt-6-luna/endpoints" \  -H "Authorization: Bearer $NUMES_API_KEY"

Validate on the client

Parse the content, validate it against the same schema, and retry or fall back when validation fails. Both examples define the schema once and send it with the request.

import jsonimport osfrom openai import OpenAIfrom pydantic import BaseModel, ValidationErrorclient = OpenAI(base_url="https://api.numes.ai/api/v1", api_key=os.environ["NUMES_API_KEY"])class CalendarEvent(BaseModel):    title: str    date: str    participants: list[str]    location: str | NoneSCHEMA = {    "type": "object",    "properties": {        "title": {"type": "string"},        "date": {"type": "string", "description": "ISO 8601 date, YYYY-MM-DD"},        "participants": {"type": "array", "items": {"type": "string"}},        "location": {"type": ["string", "null"]},    },    "required": ["title", "date", "participants", "location"],    "additionalProperties": False,}def extract_event(text: str, attempts: int = 2) -> CalendarEvent:    last_error: Exception | None = None    for _ in range(attempts):        response = client.chat.completions.create(            model="openai/gpt-6-luna",            messages=[                {"role": "system", "content": "Extract the calendar event from the user's message."},                {"role": "user", "content": text},            ],            response_format={                "type": "json_schema",                "json_schema": {"name": "calendar_event", "strict": True, "schema": SCHEMA},            },            max_tokens=512,        )        choice = response.choices[0]        if choice.finish_reason == "length":            last_error = RuntimeError("answer was truncated; raise max_tokens")            continue        try:            return CalendarEvent.model_validate_json(choice.message.content or "")        except ValidationError as error:            last_error = error  # valid JSON object, but not your schema    raise RuntimeError(f"no valid event after {attempts} attempts") from last_errorprint(extract_event("Design review with Ana and Tomás next Tuesday, 3 March 2026, in room 4B."))

Keep the request schema and the validation schema in one place. If you generate the JSON Schema from Pydantic or Zod, check that the generator emits a root object and only keywords your providers accept.

Stream structured output

Streaming works with response_format. JSON arrives as ordinary content fragments, so the text is incomplete until the finish chunk. Accumulate it and parse once:

stream = client.chat.completions.create(    model="openai/gpt-6-luna",    messages=[{"role": "user", "content": "List three prime numbers as {\"primes\": [...]}. Reply in JSON."}],    response_format={"type": "json_object"},    stream=True,)parts, finish_reason = [], Nonefor chunk in stream:    if not chunk.choices:        continue  # the usage-only chunk    choice = chunk.choices[0]    if choice.delta.content:        parts.append(choice.delta.content)    finish_reason = choice.finish_reason or finish_reasonif finish_reason != "length":    result = json.loads("".join(parts))

If you render partial JSON while it streams, use a tolerant incremental parser for display only, and validate the final object as above. An in-band error chunk means the object will not be completed.

Structured output with tools

response_format and tools can be combined. The JSON object check applies to the final answer, not to responses that call tools. See Tool calling.

Errors

Status	`error_type`	Cause
`503`	`provider_unavailable`	No eligible endpoint supports the requested format for this model.
`502`	`provider_unavailable`	Every attempted endpoint returned an answer that is not a JSON object, or the upstream rejected the requested output format. Matching compatible endpoints forward unfamiliar format fields; set `provider.require_parameters: true` to route only to endpoints that declare the format.

Tips

Make the root an object. Wrap lists in a property, such as {"items": [...]}.
Describe the task in the prompt as well. The schema constrains the shape; the instructions explain the meaning of each field.
Leave room for the answer. Set max_tokens high enough for the largest object you expect, and treat finish_reason: "length" as a failed result.
Keep schemas compact. The schema is part of the prompt and counts toward prompt tokens.
Use strict: true with complete schemas. Providers that support strict mode typically expect every property in required and additionalProperties: false.
Validate every result, even from strict endpoints, before acting on it.

Next steps

Tool callingLet the model call your functions with typed arguments.StreamingEvent format and in-band errors.Provider routingPin endpoints with the capabilities you need.ModelsRead supported parameters from the catalogue.