Multimodal

Media inputs travel inside ordinary Chat Completions requests as typed content parts in a user message. Media outputs come from dedicated endpoints that follow the OpenAI request shapes. All of them use the same inference key, routing preferences and billing as chat.

Capabilities

Chat completions return text. A chat request that asks for image or audio output in modalities fails with 503 provider_unavailable when no endpoint produces that output; use the dedicated image and speech endpoints instead. Video isn't accepted on any endpoint.

Content parts

A chat message's content can contain text and structured parts within the request's aggregate size and JSON-complexity limits. Send media using the shapes below so it passes acquisition, modality and resource checks. Unrecognized media forms, including video_url and native inline/file-source objects, are rejected. Extra media locators beside a supported part are also rejected.

Use user messages for media across native providers. A native adapter returns unsupported_capability for media on another role when its translation cannot preserve that content. Matching Chat Completions providers receive the admitted role and media unchanged.

curl -X POST "https://api.numes.ai/api/v1/chat/completions" \  -H "Authorization: Bearer $NUMES_API_KEY" \  -H "Content-Type: application/json" \  -d '{    "model": "openai/gpt-6-luna",    "messages": [      {        "role": "user",        "content": [          {            "type": "text",            "text": "What is in this picture?"          },          {            "type": "image_url",            "image_url": {              "url": "https://example.com/images/street.jpg"            }          }        ]      }    ]  }'

The model receives your text and your media as separate ordered lists, so refer to media by position ("the second image") rather than by where it sits between sentences.

Sending media

Providers never receive your URLs, cookies or headers; they receive only the validated media.

• Remote media must be a publicly reachable https:// URL that returns the file uncompressed with a 2xx status. URLs that can't be fetched promptly, or that point to non-public destinations, are rejected with 400.

• Inline media uses data:<media type>;base64,<standard base64>. Use image/jpeg, not image/jpg.

• The bytes must match the declared type. A file declared as image/png that is really a JPEG is rejected. A remote file served as application/octet-stream, or without a Content-Type, is accepted when its bytes are a supported format.

Base64 adds about a third to a file's size, and inline media counts toward the request body. HTTPS URLs don't, so use them for large images and PDFs. See Limits for rate limits.

Find models that support a modality

Each model row in GET /models has an architecture object:

{  "id": "author/vision-model",  "architecture": {    "modality": "text+image->text",    "input_modalities": ["text", "image"],    "output_modalities": ["text"]  },  "embeddings": false}

Filter by output with output_modalities, a comma-separated list. Values are text, image, embeddings, speech and transcription:

curl "https://api.numes.ai/api/v1/models?output_modalities=image" \  -H "Authorization: Bearer $NUMES_API_KEY"

A model row describes one representative endpoint, and endpoints for the same model can accept different inputs. Before you depend on image, file or audio input, check each endpoint's input_modalities in GET /models/{author}/{slug}/endpoints. The catalogue offers the same filters.

Routing sends a request only to endpoints that support every modality it uses, and never drops a media part to make a request fit. If no endpoint qualifies, the request fails with 503 provider_unavailable.

Billing by modality

Each endpoint prices media either in its own units or as prompt tokens. You're charged for the units in your request's final usage at the endpoint's rates. Failed requests that deliver no output aren't charged.

pricing.image and pricing.audio in GET /models show the lowest image and audio rate of a model's representative endpoint. Before a request runs, an upper bound of its cost is held against your balance; for image generation and speech, that bound covers every requested image and the longest possible audio. usage.cost is the exact charge, and GET /generation is the authoritative record. See Usage accounting.

Next steps