# Structured outputs

> Get answers as a JSON object or as JSON that follows your schema with response_format, and validate the result.

Source: https://0.0.0.0:3000/docs/features/structured-outputs
Documentation index: https://0.0.0.0:3000/docs/llms.txt

Structured outputs make a model answer with machine-readable JSON instead of prose. Set `response_format` on a chat completion to request a JSON object or an answer that follows your JSON Schema. The format is forwarded by matching compatible adapters. Native adapters translate supported formats to the source dialect. Set `provider.require_parameters: true` to require declared support.

How closely the answer follows your schema depends on the model and endpoint, so validate the parsed result in your application.

## Choose a format

| `response_format` | Output | Routing requirement |
| --- | --- | --- |
| `{"type": "text"}` | Plain text. Same as omitting the field. | None |
| `{"type": "json_object"}` | Any JSON object. | Endpoint supports `response_format` |
| `{"type": "json_schema", "json_schema": {...}}` | A JSON object following your schema. | Endpoint supports `structured_outputs` |

Prefer `json_schema` whenever you know the shape you need: the schema reaches the provider's native control, which constrains generation far more reliably than instructions alone.

:::fields response_format
- `type` `string` required — `text`, `json_object` or `json_schema`.
- `json_schema` `object` — Required when `type` is `json_schema`, and not allowed otherwise.
  - `name` `string` required — Provider schema identifier.
  - `description` `string` — What the output represents.
  - `schema` `object` required — The JSON Schema. Its root should describe an object.
  - `strict` `boolean` — Asks the provider to follow the schema exactly, where the provider supports strict mode.
:::

Matching compatible adapters forward unfamiliar format and schema fields. The upstream validates their semantics.

## Request a JSON schema

```request
{
  "method": "POST",
  "path": "/chat/completions",
  "body": {
    "model": "openai/gpt-6-luna",
    "messages": [
      {
        "role": "system",
        "content": "Extract the calendar event from the user's message."
      },
      {
        "role": "user",
        "content": "Design review with Ana and Tomás next Tuesday, 3 March 2026, in room 4B."
      }
    ],
    "response_format": {
      "type": "json_schema",
      "json_schema": {
        "name": "calendar_event",
        "description": "A single calendar event extracted from text.",
        "strict": true,
        "schema": {
          "type": "object",
          "properties": {
            "title": { "type": "string" },
            "date": { "type": "string", "description": "ISO 8601 date, YYYY-MM-DD" },
            "participants": { "type": "array", "items": { "type": "string" } },
            "location": { "type": ["string", "null"] }
          },
          "required": ["title", "date", "participants", "location"],
          "additionalProperties": false
        }
      }
    }
  }
}
```

The answer arrives as a JSON **string** in `message.content`:

```json title="Response excerpt"
{
  "choices": [
    {
      "index": 0,
      "finish_reason": "stop",
      "message": {
        "role": "assistant",
        "content": "{\"title\":\"Design review\",\"date\":\"2026-03-03\",\"participants\":[\"Ana\",\"Tomás\"],\"location\":\"Room 4B\"}"
      }
    }
  ]
}
```

For `json_object`, send `{"type": "json_object"}` and describe the fields you expect in the prompt. Some providers require the word "JSON" to appear in the messages for this mode, so mention it explicitly.

## JSON object check

When `type` is `json_object` or `json_schema`, the completed answer must parse as a **JSON object**. A top-level array, a bare value, text around the JSON or Markdown code fences all fail. Output is never repaired or rewritten. The check doesn't apply to responses with tool calls or to answers cut off at `max_tokens` (`finish_reason: "length"`), which are usually incomplete JSON.

When a buffered answer fails the check, Numes can retry on another eligible endpoint; if none succeeds, you receive `502 provider_unavailable`. On a stream the text has already reached you, so the stream ends with an in-band error chunk and the delivered output is billed. See [Streaming](/docs/features/streaming).

## Routing and support

With `provider.require_parameters: true`, structured output narrows routing to declared support. Without it, the selected upstream accepts or rejects the format.

Find models whose representative offer supports JSON schemas:

```request
{
  "method": "GET",
  "path": "/models",
  "query": { "supported_parameters": "structured_outputs" }
}
```

Support can differ between endpoints of the same model. Each endpoint lists `structured_outputs` and `response_format` in its `supported_parameters`:

```request
{
  "method": "GET",
  "path": "/models/openai/gpt-6-luna/endpoints"
}
```

## Validate on the client

Parse the content, validate it against the same schema, and retry or fall back when validation fails. Both examples define the schema once and send it with the request.

:::code-group
```python title="Python"
import json
import os
from openai import OpenAI
from pydantic import BaseModel, ValidationError

client = OpenAI(base_url="https://api.numes.ai/api/v1", api_key=os.environ["NUMES_API_KEY"])


class CalendarEvent(BaseModel):
    title: str
    date: str
    participants: list[str]
    location: str | None


SCHEMA = {
    "type": "object",
    "properties": {
        "title": {"type": "string"},
        "date": {"type": "string", "description": "ISO 8601 date, YYYY-MM-DD"},
        "participants": {"type": "array", "items": {"type": "string"}},
        "location": {"type": ["string", "null"]},
    },
    "required": ["title", "date", "participants", "location"],
    "additionalProperties": False,
}


def extract_event(text: str, attempts: int = 2) -> CalendarEvent:
    last_error: Exception | None = None
    for _ in range(attempts):
        response = client.chat.completions.create(
            model="openai/gpt-6-luna",
            messages=[
                {"role": "system", "content": "Extract the calendar event from the user's message."},
                {"role": "user", "content": text},
            ],
            response_format={
                "type": "json_schema",
                "json_schema": {"name": "calendar_event", "strict": True, "schema": SCHEMA},
            },
            max_tokens=512,
        )
        choice = response.choices[0]
        if choice.finish_reason == "length":
            last_error = RuntimeError("answer was truncated; raise max_tokens")
            continue
        try:
            return CalendarEvent.model_validate_json(choice.message.content or "")
        except ValidationError as error:
            last_error = error  # valid JSON object, but not your schema
    raise RuntimeError(f"no valid event after {attempts} attempts") from last_error


print(extract_event("Design review with Ana and Tomás next Tuesday, 3 March 2026, in room 4B."))
```

```typescript title="TypeScript"
import OpenAI from "openai";
import { z } from "zod";

const client = new OpenAI({ baseURL: "https://api.numes.ai/api/v1", apiKey: process.env.NUMES_API_KEY });

const CalendarEvent = z.object({
  title: z.string(),
  date: z.string(),
  participants: z.array(z.string()),
  location: z.string().nullable(),
});
type CalendarEvent = z.infer<typeof CalendarEvent>;

const schema = {
  type: "object",
  properties: {
    title: { type: "string" },
    date: { type: "string", description: "ISO 8601 date, YYYY-MM-DD" },
    participants: { type: "array", items: { type: "string" } },
    location: { type: ["string", "null"] },
  },
  required: ["title", "date", "participants", "location"],
  additionalProperties: false,
};

async function extractEvent(text: string, attempts = 2): Promise<CalendarEvent> {
  let lastError: unknown;
  for (let attempt = 0; attempt < attempts; attempt++) {
    const response = await client.chat.completions.create({
      model: "openai/gpt-6-luna",
      messages: [
        { role: "system", content: "Extract the calendar event from the user's message." },
        { role: "user", content: text },
      ],
      response_format: {
        type: "json_schema",
        json_schema: { name: "calendar_event", strict: true, schema },
      },
      max_tokens: 512,
    });
    const choice = response.choices[0];
    if (choice.finish_reason === "length") {
      lastError = new Error("answer was truncated; raise max_tokens");
      continue;
    }
    try {
      const parsed = CalendarEvent.safeParse(JSON.parse(choice.message.content ?? ""));
      if (parsed.success) return parsed.data;
      lastError = parsed.error; // valid JSON object, but not your schema
    } catch (error) {
      lastError = error;
    }
  }
  throw new Error(`no valid event after ${attempts} attempts: ${String(lastError)}`);
}

console.log(await extractEvent("Design review with Ana and Tomás next Tuesday, 3 March 2026, in room 4B."));
```
:::

Keep the request schema and the validation schema in one place. If you generate the JSON Schema from Pydantic or Zod, check that the generator emits a root object and only keywords your providers accept.

## Stream structured output

Streaming works with `response_format`. JSON arrives as ordinary `content` fragments, so the text is incomplete until the finish chunk. Accumulate it and parse once:

:::code-group
```python title="Python"
stream = client.chat.completions.create(
    model="openai/gpt-6-luna",
    messages=[{"role": "user", "content": "List three prime numbers as {\"primes\": [...]}. Reply in JSON."}],
    response_format={"type": "json_object"},
    stream=True,
)

parts, finish_reason = [], None
for chunk in stream:
    if not chunk.choices:
        continue  # the usage-only chunk
    choice = chunk.choices[0]
    if choice.delta.content:
        parts.append(choice.delta.content)
    finish_reason = choice.finish_reason or finish_reason

if finish_reason != "length":
    result = json.loads("".join(parts))
```

```typescript title="TypeScript"
const stream = await client.chat.completions.create({
  model: "openai/gpt-6-luna",
  messages: [{ role: "user", content: 'List three prime numbers as {"primes": [...]}. Reply in JSON.' }],
  response_format: { type: "json_object" },
  stream: true,
});

let text = "";
let finishReason: string | null = null;
for await (const chunk of stream) {
  const choice = chunk.choices[0]; // undefined on the usage-only chunk
  if (!choice) continue;
  text += choice.delta.content ?? "";
  finishReason = choice.finish_reason ?? finishReason;
}

const result = finishReason === "length" ? undefined : JSON.parse(text);
```
:::

If you render partial JSON while it streams, use a tolerant incremental parser for display only, and validate the final object as above. An in-band error chunk means the object will not be completed.

## Structured output with tools

`response_format` and `tools` can be combined. The JSON object check applies to the final answer, not to responses that call tools. See [Tool calling](/docs/features/tool-calling).

## Errors

| Status | `error_type` | Cause |
| --- | --- | --- |
| `503` | `provider_unavailable` | No eligible endpoint supports the requested format for this model. |
| `502` | `provider_unavailable` | Every attempted endpoint returned an answer that is not a JSON object, or the upstream rejected the requested output format. Matching compatible endpoints forward unfamiliar format fields; set `provider.require_parameters: true` to route only to endpoints that declare the format. |

## Tips

- **Make the root an object.** Wrap lists in a property, such as `{"items": [...]}`.
- **Describe the task in the prompt as well.** The schema constrains the shape; the instructions explain the meaning of each field.
- **Leave room for the answer.** Set `max_tokens` high enough for the largest object you expect, and treat `finish_reason: "length"` as a failed result.
- **Keep schemas compact.** The schema is part of the prompt and counts toward prompt tokens.
- **Use `strict: true` with complete schemas.** Providers that support strict mode typically expect every property in `required` and `additionalProperties: false`.
- **Validate every result**, even from strict endpoints, before acting on it.

## Next steps

:::cards
- {wrench} [Tool calling](/docs/features/tool-calling) Let the model call your functions with typed arguments.
- {waves} [Streaming](/docs/features/streaming) Event format and in-band errors.
- {network} [Provider routing](/docs/routing/provider-routing) Pin endpoints with the capabilities you need.
- {boxes} [Models](/docs/models) Read supported parameters from the catalogue.
:::