Audio inputs

Audio-capable chat models can answer questions about a call, summarize a voice memo or follow spoken instructions. Add an input_audio content part to a user message with the audio as base64 and its format. The examples use openai/gpt-6-luna; replace it with a model that has an endpoint whose input_modalities include audio. If you only need the spoken words, use transcription.

Send an audio file

{  "model": "openai/gpt-6-luna",  "messages": [    {      "role": "user",      "content": [        { "type": "text", "text": "What does the caller ask for, and what tone do they use?" },        { "type": "input_audio", "input_audio": { "data": "UklGRiQAAABXQVZFZm10...", "format": "wav" } }      ]    }  ]}

Read the file, encode it with standard base64 and pick the format that matches its container:

import base64import osfrom pathlib import Path​from openai import OpenAI​FORMATS = {".wav": "wav", ".mp3": "mp3", ".flac": "flac", ".m4a": "m4a", ".ogg": "ogg",           ".webm": "webm", ".aac": "aac", ".aiff": "aiff", ".aif": "aiff"}​def audio_part(path: str) -> dict:    file = Path(path)    data = base64.b64encode(file.read_bytes()).decode("ascii")    return {"type": "input_audio", "input_audio": {"data": data, "format": FORMATS[file.suffix.lower()]}}​client = OpenAI(base_url="https://api.numes.ai/api/v1", api_key=os.environ["NUMES_API_KEY"])​completion = client.chat.completions.create(    model="openai/gpt-6-luna",    messages=[        {            "role": "user",            "content": [                {"type": "text", "text": "Summarize this voice memo as a to-do list."},                audio_part("memo.mp3"),            ],        }    ],)print(completion.choices[0].message.content)

data is the bare base64 string: no data:audio/...;base64, prefix, and no URL. Audio is always inline, so it counts toward the 10 MiB chat body limit after base64 adds about a third. Keep the audio in one request under about 7.5 MiB, compress it (for example mono Ogg or low-bitrate MP3), or use transcription for long recordings.

The input_audio part

You can attach several clips to one message; they reach the model in the order you list them.

Supported formats

The format you declare must match the file's bytes. A mismatch, such as mp3 for an M4A file, is rejected.

Raw PCM has no header, so it's read as 24 kHz mono, and its length must be a whole number of samples. Resample other PCM to 24 kHz first, or wrap it in a WAV header.

Every clip must have a measurable duration, and all audio in one request may last at most 1 hour. Audio that can't be parsed, is truncated or exceeds the limit is rejected with 400. Some browser recordings, such as WebM files from MediaRecorder, can't be measured; re-mux them or send Ogg or WAV.

Route to audio-capable endpoints

Routing considers only endpoints whose input_modalities include audio. Check a model's endpoints first:

curl "https://api.numes.ai/api/v1/models/author/audio-model/endpoints" \  -H "Authorization: Bearer $NUMES_API_KEY"

Billing

Endpoints price input audio either per second, reported as input_audio_seconds in usage, or as prompt tokens. Text tokens and any per-request price apply as usual. pricing.audio in GET /models shows the lowest audio rate of the model's representative endpoint. See Usage accounting.

Chat audio or transcription

To produce speech, use text to speech.

Troubleshooting

Audio problems return 400 invalid_request. On chat completions the message names the failing part, such as messages[0].content[1].input_audio.

Next steps