Multimodal

Speech and transcription

Generate spoken audio from text with POST /audio/speech and transcribe recordings with POST /audio/transcriptions, using the OpenAI request shapes.

On this page

Text to speech
Request fields
Response
Failures during playback
Speech billing
Transcription
Upload a file
Send base64 JSON
Transcription fields
Transcription response
Transcription limits
Transcription billing
Troubleshooting
Next steps

	Text to speech	Transcription
Endpoint	`POST /audio/speech`	`POST /audio/transcriptions`
Input	JSON with text and a voice	Multipart upload or base64 JSON
Output	Raw MP3, WAV or PCM bytes	JSON with `text` and `usage`
Find models	`GET /models?output_modalities=speech`	`GET /models?output_modalities=transcription`

Both are available on models whose endpoints offer them; use the filters above to find them.

Speech input and returned transcript text preserve JSON-escaped control characters without stripping. Text, request and response size limits still apply.

Text to speech

POST/api/v1/audio/speechReference →

curl -X POST "https://api.numes.ai/api/v1/audio/speech" \  -H "Authorization: Bearer $NUMES_API_KEY" \  -H "Content-Type: application/json" \  -d '{    "model": "author/speech-model",    "input": "Your order has shipped and will arrive on Thursday.",    "voice": "alloy",    "response_format": "mp3"  }'

Voice names depend on the model; alloy is illustrative.

Request fields

Request body

modelstringRequired: Canonical ID of a speech model, such as author/speech-model.
inputstringRequired: Text to speak. Must not be blank; at most 4,096 bytes of UTF-8, which is fewer than 4,096 characters for most non-Latin scripts.
voicestringRequired: Voice name, at most 128 bytes, without control characters.
response_formatstringDefault mp3: mp3, wav or pcm.
speednumberDefault 1: Playback speed from 0.25 to 4.

Other fields, including instructions and stream_format, are rejected with 400. The request body is limited to 1 MiB.

Response

A successful response is 200 with the audio itself as the body, streamed as it's generated. There's no JSON, SSE or base64 wrapping.

Header	Value
`Content-Type`	`audio/mpeg`, `audio/wav`, or `audio/pcm` with `rate` and `bits` parameters, such as `audio/pcm;rate=24000;bits=16`
`X-Generation-Id`	The generation ID for looking up cost and outcome

For PCM, read the sample rate and bit depth from Content-Type rather than assuming them.

curl "https://api.numes.ai/api/v1/audio/speech" \  -H "Authorization: Bearer $NUMES_API_KEY" \  -H "Content-Type: application/json" \  -d '{"model": "author/speech-model", "input": "Your order has shipped.", "voice": "alloy"}' \  --dump-header headers.txt \  --output shipped.mp3

To start playback before generation finishes, read the body incrementally and feed it to your audio player.

Failures during playback

Errors before the first audio byte return an ordinary JSON error, such as 400, 402 or 503.

To confirm that you received the complete audio, look up the generation with the X-Generation-Id header:

curl "https://api.numes.ai/api/v1/generation?id=0f5b5c9a-3a7e-4c6f-9d7e-2b1c4a8e9f10" \  -H "Authorization: Bearer $NUMES_API_KEY"

An outcome of success means the audio is complete; any other final outcome means it stopped early. See Usage accounting for the outcomes.

Speech billing

Unit	Charged for
`input_characters`	Unicode characters in `input`; `"héllo"` is 5 characters although it is 6 bytes
`output_audio_seconds`	Seconds of audio delivered to you
Request	The endpoint's per-request price, if it has one

The audio response carries no usage object. Read the charge from total_cost in the generation record. Before the request runs, an upper bound of the cost is held, including the longest possible audio, so your balance and key limit must cover it.

Transcription

POST/api/v1/audio/transcriptionsReference →

Send the recording as a multipart/form-data upload, the way the OpenAI SDKs do, or as base64 in a JSON body. Other request content types are rejected with 400.

Upload a file

curl "https://api.numes.ai/api/v1/audio/transcriptions" \  -H "Authorization: Bearer $NUMES_API_KEY" \  -F "model=author/transcription-model" \  -F "[email protected];type=audio/mpeg" \  -F "language=en"

Send base64 JSON

Supply exactly one of audio or file; both take the same {data, format} object:

Request body

{  "model": "author/transcription-model",  "audio": { "data": "UklGRiQAAABXQVZFZm10...", "format": "wav" },  "language": "en"}

Raw PCM can't be detected from its bytes, so send it through the JSON form as pcm16 or pcm24 (24 kHz mono). See Audio inputs for every format.

Transcription fields

Multipart form fields

filefileRequired: Exactly one part named file, with a filename. WAV, MP3, FLAC, M4A, Ogg, WebM, AAC or AIFF; the format is detected from the bytes.
modelstringRequired: Canonical ID of a transcription model, such as author/transcription-model.
languagestring: Language of the audio, such as en. At most 16 bytes.
promptstring: At most 4,096 bytes. Accepted for compatibility and ignored.
response_formatstringDefault json: json or verbose_json. Both return the same shape.
temperaturenumber: Sampling temperature from 0 to 1.
timestamp_granularities[]string: word or segment; repeat the field for both. Forwarded to the model, but the response doesn't include timestamps.

A multipart request may contain at most 32 parts, each text field at most 8,192 bytes, and no other field names.

JSON body

modelstringRequired

Canonical ID of a transcription model.

audioobject

The recording. Supply exactly one of audio or file.

Show child attributes

datastringRequired: Standard base64 of the audio file, without a data: prefix.
formatstringRequired: wav, mp3, flac, m4a, ogg, webm, aac, aiff, pcm16 or pcm24.

fileobject

Alternative name for audio, with the same fields.

languagestring

As for multipart.

promptstring

As for multipart; ignored.

response_formatstringDefault json

json or verbose_json.

temperaturenumber

From 0 to 1.

timestamp_granularitiesstring[]

Up to two of word and segment.

Unknown JSON fields are rejected with 400.

Transcription response

Response

{  "text": "Thanks everyone for joining. First item on the agenda is the launch date.",  "usage": {    "type": "duration",    "seconds": 312.48,    "input_audio_seconds": 312.48,    "output_tokens": 18,    "cost": 0.031248  }}

Response body

textstring

The transcript.

usageobject

Usage for the request.

Show child attributes

type"duration": Usage is duration-based.
secondsnumber: Input audio duration, with millisecond precision.
input_audio_secondsnumber: The same duration, under the name used in other usage objects.
output_tokensinteger: Transcript output tokens.
costnumber: Total charge in your key's currency.

The response also carries X-Generation-Id. Word and segment arrays aren't returned, and text, srt and vtt response formats are rejected with 400.

Transcription limits

Limit	Value
Audio file size	8 MiB decoded
Audio duration	1 hour
Request body	25 MiB
Request timeout	10 minutes

The audio file is limited to 8 MiB for both multipart and JSON uploads: about 8.5 minutes of 128 kbps MP3, or 35 minutes at 32 kbps. For longer recordings, encode mono at a lower bitrate, or split the audio. Audio without a measurable duration is rejected with 400.

Transcription billing

Unit	Charged for
`input_audio_seconds`	Seconds of your recording, at the endpoint's audio input rate
Transcript output	`output_tokens`, at the endpoint's completion rate, where the endpoint prices it
Request	The endpoint's per-request price, if it has one

usage.cost is the exact charge, and the generation record is authoritative.

Troubleshooting

Symptom	Cause and fix
Speech 400	Blank `input`, `input` over 4,096 bytes, missing `voice`, `speed` outside 0.25–4, or an unsupported field such as `instructions`.
Speech audio ends early	The request failed after streaming began. Check `outcome` with `GET /generation`.
Speech 402	Your balance or key limit can't cover the held amount for the longest possible audio.
Transcription 400 on upload	The `file` part is missing, has no filename or appears twice, or an unknown form field was sent.
Transcription 400 for a valid file	The file part's `Content-Type` contradicts the detected format, such as `video/webm` for WebM audio. Declare the audio type, such as `audio/webm`, or use `application/octet-stream`.
Transcription 400 for a browser recording	Some browser WebM recordings can't be measured. Re-mux it or send Ogg or WAV.
Transcription 400 for a large file	The audio exceeds 8 MiB or 1 hour. Compress or split it.
503 `provider_unavailable`	No endpoint serves speech or transcription for the model under your routing preferences.

Next steps

Audio inputsSend audio to chat models for analysis.MultimodalMedia rules, limits and billing for every modality.Usage accountingRead usage, cost and generation records.API referenceFull schemas for the speech and transcription endpoints.