# OpenAI SDK

> Use the official OpenAI Python and TypeScript SDKs with Numes, including extension fields, exact costs, errors and retries.

Source: https://0.0.0.0:3000/docs/integrations/openai-sdk
Documentation index: https://0.0.0.0:3000/docs/llms.txt

The official OpenAI SDKs work with Numes unchanged: set the base URL and use an inference key.

## Set up the client

:::code-group
```python title="Python"
# pip install openai
import os
from openai import OpenAI

client = OpenAI(
    base_url="https://api.numes.ai/api/v1",
    api_key=os.environ["NUMES_API_KEY"],
    # Optional: attribute traffic to your app in Activity.
    default_headers={"HTTP-Referer": "https://app.example-store.io"},
)
```

```typescript title="TypeScript"
// npm install openai
import OpenAI from "openai";

const client = new OpenAI({
  baseURL: "https://api.numes.ai/api/v1",
  apiKey: process.env.NUMES_API_KEY,
  // Optional: attribute traffic to your app in Activity.
  defaultHeaders: { "HTTP-Referer": "https://app.example-store.io" },
});
```
:::

Create the client once and reuse it. Never use it in browser code: the key would be exposed, and the API rejects browser requests from other websites.

## Chat completions

:::code-group
```python title="Python"
completion = client.chat.completions.create(
    model="openai/gpt-6-luna",
    messages=[
        {"role": "system", "content": "You answer in one sentence."},
        {"role": "user", "content": "What does a reverse proxy do?"},
    ],
    max_tokens=256,
)

print(completion.choices[0].message.content)
print(completion.id)  # generation ID, for GET /generation and Logs
```

```typescript title="TypeScript"
const completion = await client.chat.completions.create({
  model: "openai/gpt-6-luna",
  messages: [
    { role: "system", content: "You answer in one sentence." },
    { role: "user", content: "What does a reverse proxy do?" },
  ],
  max_tokens: 256,
});

console.log(completion.choices[0]?.message.content);
console.log(completion.id); // generation ID, for GET /generation and Logs
```
:::

`max_tokens` caps the output. It doesn't reduce the amount held from your balance while the request runs; see [Credits and reservations](/docs/api-reference/limits#credits-and-reservations).

## Streaming

A stream's last data chunk carries `usage` and has an empty `choices` array, so guard before indexing:

:::code-group
```python title="Python"
stream = client.chat.completions.create(
    model="openai/gpt-6-luna",
    messages=[{"role": "user", "content": "Write a limerick about latency."}],
    stream=True,
)

for chunk in stream:
    if chunk.choices:
        print(chunk.choices[0].delta.content or "", end="", flush=True)
    if chunk.usage:
        print(f"\ncost: {chunk.usage.model_dump()['cost']}")
```

```typescript title="TypeScript"
const stream = await client.chat.completions.create({
  model: "openai/gpt-6-luna",
  messages: [{ role: "user", content: "Write a limerick about latency." }],
  stream: true,
});

for await (const chunk of stream) {
  const delta = chunk.choices[0]?.delta?.content;
  if (delta) process.stdout.write(delta);
  if (chunk.usage) {
    const { cost } = chunk.usage as OpenAI.CompletionUsage & { cost: number };
    console.log(`\ncost: ${cost}`);
  }
}
```
:::

Usage is always included, so `stream_options` isn't needed. To cancel, close the stream (`stream.close()` in Python, `stream.controller.abort()` in TypeScript). If a stream ends with an error instead of usage, read the final cost with [`GET /generation`](#look-up-a-generation). See [Streaming](/docs/features/streaming).

## Tool calling

Pick a model whose `supported_parameters` include `tools`. Send tools, run any calls, return the results and call again:

:::code-group
```python title="Python"
import json

tools = [{
    "type": "function",
    "function": {
        "name": "get_order_status",
        "description": "Look up the shipping status of an order.",
        "parameters": {
            "type": "object",
            "properties": {"order_id": {"type": "string"}},
            "required": ["order_id"],
        },
    },
}]

def get_order_status(order_id: str) -> dict:
    return {"order_id": order_id, "status": "shipped"}

messages = [{"role": "user", "content": "Where is order 4821?"}]
while True:
    response = client.chat.completions.create(
        model="openai/gpt-6-luna", messages=messages, tools=tools
    )
    message = response.choices[0].message
    if not message.tool_calls:
        print(message.content)
        break
    messages.append(message.model_dump(exclude_none=True))
    for call in message.tool_calls:
        result = get_order_status(**json.loads(call.function.arguments))
        messages.append(
            {"role": "tool", "tool_call_id": call.id, "content": json.dumps(result)}
        )
```

```typescript title="TypeScript"
import type { ChatCompletionMessageParam, ChatCompletionTool } from "openai/resources/chat/completions";

const tools: ChatCompletionTool[] = [{
  type: "function",
  function: {
    name: "get_order_status",
    description: "Look up the shipping status of an order.",
    parameters: {
      type: "object",
      properties: { order_id: { type: "string" } },
      required: ["order_id"],
    },
  },
}];

const messages: ChatCompletionMessageParam[] = [
  { role: "user", content: "Where is order 4821?" },
];

while (true) {
  const response = await client.chat.completions.create({
    model: "openai/gpt-6-luna",
    messages,
    tools,
  });
  const message = response.choices[0]!.message;
  if (!message.tool_calls?.length) {
    console.log(message.content);
    break;
  }
  messages.push(message);
  for (const call of message.tool_calls) {
    if (call.type !== "function") continue;
    const { order_id } = JSON.parse(call.function.arguments);
    messages.push({
      role: "tool",
      tool_call_id: call.id,
      content: JSON.stringify({ order_id, status: "shipped" }),
    });
  }
}
```
:::

Send the assistant message back unchanged, as above, so `reasoning_details` from reasoning models are preserved. See [Tool calling](/docs/features/tool-calling).

## Structured outputs

Request a JSON schema with `response_format`, or use the SDKs' `parse` helpers, which send the same field. Only endpoints that support structured outputs serve the request.

```python title="Python"
import json

completion = client.chat.completions.create(
    model="openai/gpt-6-luna",
    messages=[{"role": "user", "content": "Extract: Ada Lovelace, born 1815."}],
    response_format={
        "type": "json_schema",
        "json_schema": {
            "name": "person",
            "strict": True,
            "schema": {
                "type": "object",
                "properties": {
                    "name": {"type": "string"},
                    "birth_year": {"type": "integer"},
                },
                "required": ["name", "birth_year"],
                "additionalProperties": False,
            },
        },
    },
)

person = json.loads(completion.choices[0].message.content)
```

Schema adherence depends on the model and endpoint, so validate the parsed result. See [Structured outputs](/docs/features/structured-outputs).

## Send extension fields

Fields outside OpenAI's API, such as `provider`, `reasoning` and `session_id`, go in `extra_body` in Python. In TypeScript, spread them into the request, which bypasses the SDK's type check. Extra headers go in `extra_headers` or the second argument's `headers`.

:::code-group
```python title="Python"
completion = client.chat.completions.create(
    model="openai/gpt-6-luna",
    messages=[{"role": "user", "content": "Summarize this ticket."}],
    extra_body={
        "provider": {"sort": "throughput", "data_collection": "deny"},
        "reasoning": {"effort": "low"},
    },
    extra_headers={"X-Session-Id": "ticket-4821"},
)
```

```typescript title="TypeScript"
const extensions = {
  provider: { sort: "throughput", data_collection: "deny" },
  reasoning: { effort: "low" },
};

const completion = await client.chat.completions.create(
  {
    model: "openai/gpt-6-luna",
    messages: [{ role: "user", content: "Summarize this ticket." }],
    ...extensions,
  },
  { headers: { "X-Session-Id": "ticket-4821" } },
);
```
:::

Any field the API doesn't define returns `400`; see [Parameters](/docs/api-reference/parameters).

## Read the exact cost

`usage.cost` is the charge in the key's currency: dollars for USD keys, satoshis for BTC keys. The SDK types don't declare it, but it's present on the object:

:::code-group
```python title="Python"
from decimal import Decimal

cost = Decimal(str(completion.usage.model_dump()["cost"]))
```

```typescript title="TypeScript"
const { cost } = completion.usage as OpenAI.CompletionUsage & { cost: number };
```
:::

Add costs up in a decimal type, not floats. See [Usage accounting](/docs/account/usage-accounting).

## Look up a generation

Use the SDK's generic request methods for endpoints without a dedicated method, such as [`GET /generation`](/docs/api-reference/chat/get-generation):

:::code-group
```python title="Python"
import httpx

response = client.get(
    "/generation",
    cast_to=httpx.Response,
    options={"params": {"id": completion.id}},
)
generation = response.json()["data"]
print(generation["outcome"], generation["total_cost"])
```

```typescript title="TypeScript"
const { data: generation } = (await client.get("/generation", {
  query: { id: completion.id },
})) as { data: { outcome: string; total_cost: number } };

console.log(generation.outcome, generation.total_cost);
```
:::

To read response headers such as `X-Generation-Id` and `X-Request-Id` on any call, use `client.chat.completions.with_raw_response.create(...)` in Python or `.withResponse()` in TypeScript.

## Embeddings

:::code-group
```python title="Python"
result = client.embeddings.create(
    model="author/embedding-model",
    input=["first document", "second document"],
    encoding_format="float",
)
vectors = [item.embedding for item in result.data]
```

```typescript title="TypeScript"
const result = await client.embeddings.create({
  model: "author/embedding-model",
  input: ["first document", "second document"],
  encoding_format: "float",
});
const vectors = result.data.map((item) => item.embedding);
```
:::

Recent SDK versions request `base64` vectors by default. Set `encoding_format="float"` so the result doesn't depend on an endpoint's base64 support. See [Embeddings](/docs/multimodal/embeddings).

## Speech and transcription

:::code-group
```python title="Python"
with client.audio.speech.with_streaming_response.create(
    model="author/speech-model",
    voice="alloy",
    input="Your order has shipped.",
    response_format="mp3",
) as response:
    response.stream_to_file("shipped.mp3")

with open("meeting.mp3", "rb") as audio:
    transcript = client.audio.transcriptions.create(
        model="author/transcription-model",
        file=audio,
    )
print(transcript.text)
```

```typescript title="TypeScript"
import fs from "node:fs";

const speech = await client.audio.speech.create({
  model: "author/speech-model",
  voice: "alloy",
  input: "Your order has shipped.",
  response_format: "mp3",
});
await fs.promises.writeFile("shipped.mp3", Buffer.from(await speech.arrayBuffer()));

const transcript = await client.audio.transcriptions.create({
  model: "author/transcription-model",
  file: fs.createReadStream("meeting.mp3"),
});
console.log(transcript.text);
```
:::

Speech supports `mp3`, `wav` and `pcm`, and voices depend on the model. Transcription returns `json` or `verbose_json`. See [Speech](/docs/multimodal/speech).

## Image generation

Image generation is served at `POST /images`, not OpenAI's `/images/generations`, so `client.images.generate` does not reach it. Use the generic request method:

:::code-group
```python title="Python"
import base64
import httpx

response = client.post(
    "/images",
    cast_to=httpx.Response,
    body={"model": "author/image-model", "prompt": "A lighthouse at dusk", "size": "1024x1024"},
)
image = response.json()["data"][0]
with open("lighthouse.png", "wb") as file:
    file.write(base64.b64decode(image["b64_json"]))
```

```typescript title="TypeScript"
import fs from "node:fs";

const images = (await client.post("/images", {
  body: { model: "author/image-model", prompt: "A lighthouse at dusk", size: "1024x1024" },
})) as { data: { b64_json: string; media_type: string }[] };

await fs.promises.writeFile("lighthouse.png", Buffer.from(images.data[0]!.b64_json, "base64"));
```
:::

Images are returned as base64. See [Image generation](/docs/multimodal/image-generation).

## Handle errors

Errors use the envelope `{"error": {"code", "message", "metadata": {"error_type"}}}`. The SDKs raise an exception by status:

| Status | Python exception | TypeScript class |
| --- | --- | --- |
| 400 | `BadRequestError` | `BadRequestError` |
| 401 | `AuthenticationError` | `AuthenticationError` |
| 402, 408, 413, 415 | `APIStatusError` | `APIError` |
| 403 | `PermissionDeniedError` | `PermissionDeniedError` |
| 404 | `NotFoundError` | `NotFoundError` |
| 429 | `RateLimitError` | `RateLimitError` |
| 5xx | `InternalServerError` | `InternalServerError` |
| Network failure | `APIConnectionError` | `APIConnectionError` |

:::code-group
```python title="Python"
import openai

try:
    completion = client.chat.completions.create(
        model="openai/gpt-6-luna",
        messages=[{"role": "user", "content": "Hello"}],
    )
except openai.APIStatusError as error:
    body = error.body if isinstance(error.body, dict) else {}
    error_type = body.get("metadata", {}).get("error_type")
    print(error.status_code, error_type, error.request_id)
    raise
```

```typescript title="TypeScript"
try {
  await client.chat.completions.create({
    model: "openai/gpt-6-luna",
    messages: [{ role: "user", content: "Hello" }],
  });
} catch (error) {
  if (error instanceof OpenAI.APIError) {
    const errorType = (error.error as { metadata?: { error_type?: string } } | undefined)
      ?.metadata?.error_type;
    console.error(error.status, errorType, error.requestID);
  }
  throw error;
}
```
:::

Quote the request ID when you contact support. An error after streaming has started arrives as a final chunk with `finish_reason: "error"`, not as an exception. See [Errors](/docs/api-reference/errors).

## Retries and timeouts

Both SDKs retry twice by default, with backoff, on connection errors, `408`, `409`, `429` and `5xx`, and honor `Retry-After`. They never retry `402`, and never retry a stream that has already started, so a retry can't charge you twice for delivered output.

Inference requests end after 10 minutes, so set the client timeout slightly above that:

:::code-group
```python title="Python"
client = OpenAI(
    base_url="https://api.numes.ai/api/v1",
    api_key=os.environ["NUMES_API_KEY"],
    max_retries=3,
    timeout=630.0,  # seconds
)
```

```typescript title="TypeScript"
const client = new OpenAI({
  baseURL: "https://api.numes.ai/api/v1",
  apiKey: process.env.NUMES_API_KEY,
  maxRetries: 3,
  timeout: 630_000, // milliseconds
});
```
:::

Don't set `X-Request-Id` in the client's default headers: retries would resend it, and a reused ID returns `400`. Read the server-assigned ID from the error or the raw response instead.

## Next steps

:::cards
- {plug} [Compatibility](/docs/api-reference/compatibility) Every difference from OpenRouter.
- {network} [Provider routing](/docs/routing/provider-routing) The `provider` object in detail.
- {wrench} [Tool calling](/docs/features/tool-calling) Tool choice, parallel calls and hosted tools.
- {receipt} [Usage accounting](/docs/account/usage-accounting) What `usage` and `cost` mean.
- {alert} [Errors](/docs/api-reference/errors) Every error type and what to do.
:::