Streaming

Set stream: true on a chat completion to receive output as it's generated. The response is a Server-Sent Events (SSE) stream of OpenAI-compatible chat.completion.chunk objects that ends with a usage chunk carrying the request's cost.

Text, reasoning, argument fragments, transcripts and citation text preserve control characters through JSON escaping. Decode each JSON event before using its text, and escape it for HTML or terminal display. Response size and structure limits still apply.

Citation URLs may use HTTP or HTTPS and are limited to 8,192 UTF-8 bytes. A citation is a reference; receiving it does not fetch its contents.

Start a stream

Send the usual request body with stream: true. Choose Try it to watch the events arrive.

curl -N -X POST "https://api.numes.ai/api/v1/chat/completions" \  -H "Authorization: Bearer $NUMES_API_KEY" \  -H "Content-Type: application/json" \  -d '{    "model": "openai/gpt-6-luna",    "stream": true,    "messages": [      {        "role": "user",        "content": "Explain Server-Sent Events in two sentences."      }    ]  }'

A successful stream returns 200 with Content-Type: text/event-stream, Cache-Control: no-cache, X-Request-Id and X-Generation-Id. The generation ID is also every chunk's id; keep it to look up the cost later.

data: {"id":"6f1c2b8e-3d4a-4f5b-9c7d-2e8a1b0c9d3f","object":"chat.completion.chunk","created":1790000000,"model":"openai/gpt-6-luna","choices":[{"index":0,"delta":{"role":"assistant","content":"Server-Sent"},"finish_reason":null}]}​data: {"id":"6f1c2b8e-3d4a-4f5b-9c7d-2e8a1b0c9d3f","object":"chat.completion.chunk","created":1790000000,"model":"openai/gpt-6-luna","choices":[{"index":0,"delta":{"content":" Events keep one HTTP response open"},"finish_reason":null}]}​data: {"id":"6f1c2b8e-3d4a-4f5b-9c7d-2e8a1b0c9d3f","object":"chat.completion.chunk","created":1790000000,"model":"openai/gpt-6-luna","choices":[{"index":0,"delta":{},"finish_reason":"stop"}]}​data: {"id":"6f1c2b8e-3d4a-4f5b-9c7d-2e8a1b0c9d3f","object":"chat.completion.chunk","created":1790000000,"model":"openai/gpt-6-luna","choices":[],"usage":{"prompt_tokens":16,"completion_tokens":38,"total_tokens":54,"cost":0.000017,"cost_details":{"upstream_inference_cost":0.000017}}}​data: [DONE]

A successful stream always arrives in this order:

1. Output chunks. The first chunk's delta normally carries role: "assistant" along with its content; there's no separate role-only chunk.

2. Finish chunk. delta is empty, or carries a reasoning terminator, and finish_reason is set. See Finish reasons.

3. Usage chunk. choices is an empty array and usage holds the final usage and cost.

4. data: [DONE].

Delta fields

A delta carries only the fields produced by that event:

Reasoning deltas

Each visible reasoning fragment appears twice in the same chunk: as delta.reasoning and as a reasoning.text entry in delta.reasoning_details.

data: {"id":"6f1c2b8e-3d4a-4f5b-9c7d-2e8a1b0c9d3f","object":"chat.completion.chunk","created":1790000000,"model":"openai/gpt-6-luna","choices":[{"index":0,"delta":{"role":"assistant","reasoning":"The user wants brevity.","reasoning_details":[{"type":"reasoning.text","text":"The user wants brevity."}]},"finish_reason":null}]}

• When a provider signs a reasoning block after its text has streamed, a later reasoning_details entry carries the signature (and id or format) without repeating the text. Accumulate the entries in order to rebuild the signed block.

• The finish chunk has an empty delta. Provider continuation details are preserved without adding an unsigned terminator.

• reasoning.encrypted and reasoning.summary entries can also appear. Send them back unchanged.

• With reasoning: {"exclude": true}, neither representation is sent, so expect a longer pause before the first chunk.

Reasoning covers the controls and continuation.

Tool call deltas

Function calls stream in delta.tool_calls. Accumulate them by each entry's index, not by position in the array.

data: {"id":"6f1c2b8e-3d4a-4f5b-9c7d-2e8a1b0c9d3f","object":"chat.completion.chunk","created":1790000000,"model":"openai/gpt-6-luna","choices":[{"index":0,"delta":{"role":"assistant","tool_calls":[{"index":0,"id":"call_7f3a9c","type":"function","function":{"name":"get_weather"}}]},"finish_reason":null}]}​data: {"id":"6f1c2b8e-3d4a-4f5b-9c7d-2e8a1b0c9d3f","object":"chat.completion.chunk","created":1790000000,"model":"openai/gpt-6-luna","choices":[{"index":0,"delta":{"tool_calls":[{"index":0,"function":{"arguments":"{\"city\":\"Lisbon\"}"}}]},"finish_reason":null}]}​data: {"id":"6f1c2b8e-3d4a-4f5b-9c7d-2e8a1b0c9d3f","object":"chat.completion.chunk","created":1790000000,"model":"openai/gpt-6-luna","choices":[{"index":0,"delta":{},"finish_reason":"tool_calls"}]}

• The fragment that opens a call carries id, type: "function" and function.name. Later fragments for the same index may omit them.

• Some endpoints send a call's full arguments in one fragment; others split them. Append fragments in arrival order; fragments of different calls can interleave.

• Parse arguments and run tools only after the finish chunk. Some endpoints report stop instead of tool_calls, so check for accumulated calls too.

See Tool calling for the full loop.

Final usage chunk

Every successful stream ends with a usage chunk. You don't need stream_options: {"include_usage": true}: it's accepted and ignored, as is usage: {"include": true}, and setting either to false doesn't suppress the chunk.

usage.cost is the final charge in your key's currency (satoshis for BTC keys). With X-OpenRouter-Metadata: enabled, the usage chunk also carries openrouter_metadata. Usage accounting explains every usage field.

A stream that's cancelled or ends with an error has no usage chunk; read its cost from GET /generation with the generation ID.

Cancel a stream

Close the connection to stop the request; it's the only cancellation signal. With the OpenAI SDKs, call stream.close() in Python or stream.controller.abort() in TypeScript.

A cancelled stream has no usage chunk. GET /generation is the authoritative record of what was delivered and charged; it can show outcome: "in_progress" briefly while the request finalizes.

curl "https://api.numes.ai/api/v1/generation?id=6f1c2b8e-3d4a-4f5b-9c7d-2e8a1b0c9d3f" \  -H "Authorization: Bearer $NUMES_API_KEY"

Encrypted models differ: after output starts, closing the connection stops delivery to you, but the request runs to completion and is billed on the provider's final usage.

Errors

Errors before the first chunk

If the request fails before any chunk is written, you get an ordinary non-200 response with the JSON error envelope and no event stream: for example 400 for an invalid body, 402 for insufficient credits, 429 for a rate limit or 503 when no endpoint can serve the request. Before any output, the request may already have been retried on other endpoints. Check the status before you parse events.

Errors after streaming starts

Once the stream has started, the status is already 200. A later failure arrives as one final chunk with a top-level error object and finish_reason: "error", followed by data: [DONE]. No finish chunk or usage chunk is sent.

data: {"id":"6f1c2b8e-3d4a-4f5b-9c7d-2e8a1b0c9d3f","object":"chat.completion.chunk","created":1790000004,"model":"openai/gpt-6-luna","error":{"code":503,"message":"provider became unavailable","metadata":{"error_type":"provider_unavailable"}},"choices":[{"index":0,"delta":{"content":""},"finish_reason":"error"}]}​data: [DONE]

Server interruption is distinct from your own cancellation: the former uses 503 provider_unavailable, the latter 408 cancelled. A client retry is a new request; handle partial output and tool effects before restarting. See the retry guidance.

error.code is the HTTP status the error would have had, and error.metadata.error_type is the stable category:

Errors from your account's request policy can also arrive in-band with the codes listed in Errors and debugging.

Output never switches to another provider once it has reached you. The failed generation may have a charge; check its generation record. If your client retries, the new request has separate usage and cost, and must handle the partial response as described above.

A stream can also end without [DONE] when the connection drops. Treat a missing terminator as an incomplete response and check the generation record.

Timeouts

A request, including its whole stream, has a 10-minute time limit. Before any output it fails with a 408 timeout error response; after output starts you get the in-band timeout chunk.

The stream has no keep-alive frames, so long silent gaps are normal: before the first chunk, while a model reasons, or while a hosted tool runs. Set your client's read timeout to at least the request limit, and make sure proxies between you and the API don't close idle connections sooner.

Stream with the OpenAI SDK

The official SDKs parse the stream for you. Fields outside the OpenAI types, such as reasoning and usage.cost, are still present on each chunk. An in-band error chunk raises an APIError.

import osfrom openai import OpenAI, APIError​client = OpenAI(    base_url="https://api.numes.ai/api/v1",    api_key=os.environ["NUMES_API_KEY"],    timeout=360,  # seconds; at least the request time limit)​stream = client.chat.completions.create(    model="openai/gpt-6-luna",    messages=[{"role": "user", "content": "Explain Server-Sent Events in two sentences."}],    stream=True,)​generation_id, usage = None, Nonetry:    for chunk in stream:        generation_id = chunk.id        if chunk.usage is not None:            usage = chunk.usage  # usage chunk; choices is empty        if chunk.choices and chunk.choices[0].delta.content:            print(chunk.choices[0].delta.content, end="", flush=True)except APIError as error:    print(f"\nstream failed: {error.message}")finally:    stream.close()​if usage is not None:    print(f"\ncost: {getattr(usage, 'cost', None)}")else:    print(f"\nno usage chunk; look up generation {generation_id}")

Wire format

Read this section when you write or debug your own parser.

Frame syntax

• Each event is one data: line holding a JSON object with no line breaks, followed by an empty line: data: {...}\n\n.

• Frames use only the data field; there are no event:, id: or retry: fields.

• Each frame is flushed as soon as it's written.

• The stream ends with the literal frame data: [DONE], which isn't JSON.

• Lines starting with : are SSE comments. The API doesn't send them, but your parser must ignore them.

• JSON key order inside a chunk isn't significant.

Chunk shape

type ChatCompletionChunk = {  id: string; // generation ID, constant for the stream  object: "chat.completion.chunk";  created: number; // Unix seconds  model: string;  choices: [{ index: 0; delta: Delta; finish_reason: string | null }] | []; // [] on the usage chunk  usage?: Usage; // usage chunk only  openrouter_metadata?: Record<string, unknown>; // usage chunk, when requested  error?: { code: number; message: string; metadata: { error_type: string } }; // error chunk only};​type Delta = {  role?: "assistant"; // first chunk only  content?: string;  reasoning?: string;  reasoning_details?: ReasoningDetail[];  tool_calls?: {    index: number;    id?: string; // first fragment of a call    type?: "function";    function: { name?: string; arguments?: string };  }[];  annotations?: Annotation[];  artifacts?: { type: "file" | "image" | "audio"; filename: string; media_type: string; data: string }[];  images?: { index: number; type: "image_url"; image_url: { url: string } }[];  audio?: { data: string; transcript: string; format: string };};

Usage, ReasoningDetail and Annotation have the same shapes as in a buffered response; see Create chat completion. Chunks have no provider field; send X-OpenRouter-Metadata: enabled to get the serving provider on the usage chunk.

Image streams

POST /images with stream: true uses the same frame syntax, Content-Type: text/event-stream and X-Generation-Id. Each frame is a JSON object with a type:

data: {"type":"image_generation.partial","index":0,"b64_json":"iVBORw0KGgo...","media_type":"image/png"}​data: {"type":"image_generation.completed","index":0,"b64_json":"iVBORw0KGgo...","media_type":"image/png"}​data: {"type":"image_generation.usage","usage":{"input_images":0,"output_images":1,"input_audio_seconds":0,"output_audio_seconds":0,"input_characters":0,"prompt_tokens":0,"completion_tokens":0,"total_tokens":0,"cost":0.04}}​data: [DONE]

index identifies the image when you request several with n. Images are always base64; there are no image URLs. An image error frame differs from the chat shape: error.code is the error type and error.status is the HTTP status.

data: {"type":"image_generation.error","error":{"code":"provider_unavailable","message":"provider became unavailable","status":503}}​data: [DONE]

Speech streams

POST /audio/speech doesn't use SSE. It streams raw audio bytes with the audio format as Content-Type and X-Generation-Id set when the first bytes are sent; write the body straight to a file or player. Errors before the first byte use the JSON error envelope. After audio starts there's no in-band error: a failure ends the response early. Check the generation's outcome with GET /generation to confirm a complete result.

Write a robust parser

1. Check the status first. Only a 200 with Content-Type: text/event-stream is a stream; anything else is a JSON error.

2. Buffer across reads. A read can end mid-line or mid-character. Decode UTF-8 with a streaming decoder and split on \n or \r\n only after buffering.

3. An empty line ends an event. Join its data: lines with \n.

4. Ignore everything else, including : comment lines and fields other than data.

5. Stop at [DONE] without passing it to a JSON parser.

6. Check every chunk for error before reading choices, and remember the usage chunk has choices: [].

7. Reassemble tool calls by index, and read one reasoning representation.

import jsonimport os​import requests​​def sse_data(lines):    """Yield the data payload of each SSE event."""    data = []    for line in lines:        if line == "":  # a blank line ends the event            if data:                yield "\n".join(data)                data = []        elif not line.startswith(":"):  # skip comments            field, _, value = line.partition(":")            if field == "data":                data.append(value[1:] if value.startswith(" ") else value)    if data:        yield "\n".join(data)​​response = requests.post(    "https://api.numes.ai/api/v1/chat/completions",    headers={"Authorization": f"Bearer {os.environ['NUMES_API_KEY']}"},    json={        "model": "openai/gpt-6-luna",        "stream": True,        "messages": [{"role": "user", "content": "What is the weather in Oslo?"}],    },    stream=True,    timeout=(10, 360),  # connect, read)if response.status_code != 200:    raise RuntimeError(response.json()["error"])response.encoding = "utf-8"  # event streams are UTF-8​text, calls, usage = [], {}, Nonetry:    for payload in sse_data(response.iter_lines(decode_unicode=True)):        if payload == "[DONE]":            break        chunk = json.loads(payload)        if "error" in chunk:            raise RuntimeError(f"stream failed: {chunk['error']} (generation {chunk['id']})")        if chunk.get("usage"):            usage = chunk["usage"]        for choice in chunk["choices"]:            delta = choice["delta"]            text.append(delta.get("content") or "")            for call in delta.get("tool_calls", []):                slot = calls.setdefault(call["index"], {"id": None, "name": None, "arguments": ""})                slot["id"] = call.get("id", slot["id"])                function = call.get("function", {})                slot["name"] = function.get("name", slot["name"])                slot["arguments"] += function.get("arguments", "")finally:    response.close()  # closing early cancels the request​print("".join(text))for call in calls.values():    print(call["name"], json.loads(call["arguments"] or "{}"))print("cost:", usage["cost"] if usage else "unknown")

Next steps