Integrations
OpenAI SDK
Use the official OpenAI Python and TypeScript SDKs with Numes, including extension fields, exact costs, errors and retries.
The official OpenAI SDKs work with Numes unchanged: set the base URL and use an inference key.
Set up the client
# pip install openaiimport osfrom openai import OpenAIclient = OpenAI( base_url="https://api.numes.ai/api/v1", api_key=os.environ["NUMES_API_KEY"], # Optional: attribute traffic to your app in Activity. default_headers={"HTTP-Referer": "https://app.example-store.io"},)Create the client once and reuse it. Never use it in browser code: the key would be exposed, and the API rejects browser requests from other websites.
Chat completions
completion = client.chat.completions.create( model="openai/gpt-6-luna", messages=[ {"role": "system", "content": "You answer in one sentence."}, {"role": "user", "content": "What does a reverse proxy do?"}, ], max_tokens=256,)print(completion.choices[0].message.content)print(completion.id) # generation ID, for GET /generation and Logsmax_tokens caps the output. It doesn't reduce the amount held from your balance while the request runs; see Credits and reservations.
Streaming
A stream's last data chunk carries usage and has an empty choices array, so guard before indexing:
stream = client.chat.completions.create( model="openai/gpt-6-luna", messages=[{"role": "user", "content": "Write a limerick about latency."}], stream=True,)for chunk in stream: if chunk.choices: print(chunk.choices[0].delta.content or "", end="", flush=True) if chunk.usage: print(f"\ncost: {chunk.usage.model_dump()['cost']}")Usage is always included, so stream_options isn't needed. To cancel, close the stream (stream.close() in Python, stream.controller.abort() in TypeScript). If a stream ends with an error instead of usage, read the final cost with GET /generation. See Streaming.
Tool calling
Pick a model whose supported_parameters include tools. Send tools, run any calls, return the results and call again:
import jsontools = [{ "type": "function", "function": { "name": "get_order_status", "description": "Look up the shipping status of an order.", "parameters": { "type": "object", "properties": {"order_id": {"type": "string"}}, "required": ["order_id"], }, },}]def get_order_status(order_id: str) -> dict: return {"order_id": order_id, "status": "shipped"}messages = [{"role": "user", "content": "Where is order 4821?"}]while True: response = client.chat.completions.create( model="openai/gpt-6-luna", messages=messages, tools=tools ) message = response.choices[0].message if not message.tool_calls: print(message.content) break messages.append(message.model_dump(exclude_none=True)) for call in message.tool_calls: result = get_order_status(**json.loads(call.function.arguments)) messages.append( {"role": "tool", "tool_call_id": call.id, "content": json.dumps(result)} )Send the assistant message back unchanged, as above, so reasoning_details from reasoning models are preserved. See Tool calling.
Structured outputs
Request a JSON schema with response_format, or use the SDKs' parse helpers, which send the same field. Only endpoints that support structured outputs serve the request.
import jsoncompletion = client.chat.completions.create( model="openai/gpt-6-luna", messages=[{"role": "user", "content": "Extract: Ada Lovelace, born 1815."}], response_format={ "type": "json_schema", "json_schema": { "name": "person", "strict": True, "schema": { "type": "object", "properties": { "name": {"type": "string"}, "birth_year": {"type": "integer"}, }, "required": ["name", "birth_year"], "additionalProperties": False, }, }, },)person = json.loads(completion.choices[0].message.content)Schema adherence depends on the model and endpoint, so validate the parsed result. See Structured outputs.
Send extension fields
Fields outside OpenAI's API, such as provider, reasoning and session_id, go in extra_body in Python. In TypeScript, spread them into the request, which bypasses the SDK's type check. Extra headers go in extra_headers or the second argument's headers.
completion = client.chat.completions.create( model="openai/gpt-6-luna", messages=[{"role": "user", "content": "Summarize this ticket."}], extra_body={ "provider": {"sort": "throughput", "data_collection": "deny"}, "reasoning": {"effort": "low"}, }, extra_headers={"X-Session-Id": "ticket-4821"},)Any field the API doesn't define returns 400; see Parameters.
Read the exact cost
usage.cost is the charge in the key's currency: dollars for USD keys, satoshis for BTC keys. The SDK types don't declare it, but it's present on the object:
from decimal import Decimalcost = Decimal(str(completion.usage.model_dump()["cost"]))Add costs up in a decimal type, not floats. See Usage accounting.
Look up a generation
Use the SDK's generic request methods for endpoints without a dedicated method, such as GET /generation:
import httpxresponse = client.get( "/generation", cast_to=httpx.Response, options={"params": {"id": completion.id}},)generation = response.json()["data"]print(generation["outcome"], generation["total_cost"])To read response headers such as X-Generation-Id and X-Request-Id on any call, use client.chat.completions.with_raw_response.create(...) in Python or .withResponse() in TypeScript.
Embeddings
result = client.embeddings.create( model="author/embedding-model", input=["first document", "second document"], encoding_format="float",)vectors = [item.embedding for item in result.data]Recent SDK versions request base64 vectors by default. Set encoding_format="float" so the result doesn't depend on an endpoint's base64 support. See Embeddings.
Speech and transcription
with client.audio.speech.with_streaming_response.create( model="author/speech-model", voice="alloy", input="Your order has shipped.", response_format="mp3",) as response: response.stream_to_file("shipped.mp3")with open("meeting.mp3", "rb") as audio: transcript = client.audio.transcriptions.create( model="author/transcription-model", file=audio, )print(transcript.text)Speech supports mp3, wav and pcm, and voices depend on the model. Transcription returns json or verbose_json. See Speech.
Image generation
Image generation is served at POST /images, not OpenAI's /images/generations, so client.images.generate does not reach it. Use the generic request method:
import base64import httpxresponse = client.post( "/images", cast_to=httpx.Response, body={"model": "author/image-model", "prompt": "A lighthouse at dusk", "size": "1024x1024"},)image = response.json()["data"][0]with open("lighthouse.png", "wb") as file: file.write(base64.b64decode(image["b64_json"]))Images are returned as base64. See Image generation.
Handle errors
Errors use the envelope {"error": {"code", "message", "metadata": {"error_type"}}}. The SDKs raise an exception by status:
| Status | Python exception | TypeScript class |
|---|---|---|
| 400 | BadRequestError | BadRequestError |
| 401 | AuthenticationError | AuthenticationError |
| 402, 408, 413, 415 | APIStatusError | APIError |
| 403 | PermissionDeniedError | PermissionDeniedError |
| 404 | NotFoundError | NotFoundError |
| 429 | RateLimitError | RateLimitError |
| 5xx | InternalServerError | InternalServerError |
| Network failure | APIConnectionError | APIConnectionError |
import openaitry: completion = client.chat.completions.create( model="openai/gpt-6-luna", messages=[{"role": "user", "content": "Hello"}], )except openai.APIStatusError as error: body = error.body if isinstance(error.body, dict) else {} error_type = body.get("metadata", {}).get("error_type") print(error.status_code, error_type, error.request_id) raiseQuote the request ID when you contact support. An error after streaming has started arrives as a final chunk with finish_reason: "error", not as an exception. See Errors.
Retries and timeouts
Both SDKs retry twice by default, with backoff, on connection errors, 408, 409, 429 and 5xx, and honor Retry-After. They never retry 402, and never retry a stream that has already started, so a retry can't charge you twice for delivered output.
Inference requests end after 10 minutes, so set the client timeout slightly above that:
client = OpenAI( base_url="https://api.numes.ai/api/v1", api_key=os.environ["NUMES_API_KEY"], max_retries=3, timeout=630.0, # seconds)Don't set X-Request-Id in the client's default headers: retries would resend it, and a reused ID returns 400. Read the server-assigned ID from the error or the raw response instead.