Provider routing

A model on Numes is usually served by several independent provider endpoints, each with its own price, speed, location and data policy. For every request, routing decides which endpoints are eligible, orders them and tries them in turn until one delivers output. The provider object changes that behavior.

The provider object

Add provider to a chat completion, embedding or image generation request. Every field is optional; unknown fields are rejected with 400 invalid_request. Speech and transcription requests use default routing.

{  "model": "openai/gpt-6-luna",  "messages": [{ "role": "user", "content": "Hello" }],  "provider": {    "sort": "throughput",    "ignore": ["example-provider"],    "max_price": { "prompt": 0.000001, "completion": 0.000004 }  }}

List entries must be 1 to 256 characters. Object-form sort, percentile thresholds and other OpenRouter routing fields not listed here aren't supported.

Default routing

With no provider object, each request goes through three stages.

1. Eligibility

An endpoint is a candidate only when every one of these passes. They're checked for each request, and again before each fallback:

If no endpoint passes, the request fails with 503 and error.metadata.error_type provider_unavailable. An approved offer keeps serving while a replacement revision or price change is under review.

2. Price band

Routing estimates what this specific request would cost on each eligible endpoint, from your prompt, max_tokens and any per-request or media charges. An endpoint with cheap prompts and expensive completions can therefore rank differently for a long prompt than for a long answer. Endpoints whose estimate is close to the lowest form the price band.

This estimate is separate from the hold on your balance; see Credits.

3. Order within the band

Inside the band, routing balances price, reliability and performance across the eligible endpoints, with randomization to spread load. New offers receive some traffic so they can build a track record. Endpoints outside the band follow in ascending order of estimated price.

Routing tries the first endpoint. If it fails before any output, the remaining endpoints are re-checked and the next one is tried, up to a limited number of tries. See Fallbacks and reliability.

Session stickiness

Consecutive requests from the same conversation go to the same endpoint while it's still eligible, which improves prompt-cache hits. Stickiness is keyed on prompt_cache_key, then session_id, then the X-Session-Id header. It never overrides eligibility and applies only within the price band, and it's off whenever you set order or sort. See Sessions.

Selectors

order, only and ignore accept two kinds of selector:

Selectors are stable routing slugs that don't change when a provider renames itself. Matching is case-insensitive. Display names such as Example Provider don't match.

Each endpoint's tag in the endpoints API is its selector; the part before the slash is the provider slug:

curl "https://api.numes.ai/api/v1/models/openai/gpt-6-luna/endpoints" \  -H "Authorization: Bearer $NUMES_API_KEY"

The same values appear as provider_slug and endpoint in generation metadata, so you can copy the selector of an endpoint that served you well.

Order and fallbacks

order moves the listed endpoints to the front, in list order. A provider slug ranks all of that provider's endpoints at its position. Unlisted endpoints still follow as fallbacks.

curl -X POST "https://api.numes.ai/api/v1/chat/completions" \  -H "Authorization: Bearer $NUMES_API_KEY" \  -H "Content-Type: application/json" \  -d '{    "model": "openai/gpt-6-luna",    "messages": [      {        "role": "user",        "content": "Summarize the plot of Hamlet in two sentences."      }    ],    "provider": {      "order": [        "example-provider",        "another-provider/fast-gpu"      ]    }  }'

Setting order turns off the price band, the balanced order and session stickiness. Unlisted endpoints follow in a fixed order. Add sort to order endpoints that share a position, such as several endpoints of a listed provider, and the unlisted remainder. A sort never moves an endpoint ahead of one listed earlier.

{  "provider": {    "order": ["example-provider"],    "sort": "price"  }}

allow_fallbacks: false limits the request to one try. With order, unlisted endpoints are also dropped, so that try goes to the first listed endpoint that's eligible. If none is eligible, the request fails with 503; if it fails before output, there's no second try. See Turn fallbacks off.

Allow and deny lists

only restricts routing to matching endpoints; ignore removes matching endpoints. Both use the same selectors and leave ordering to the default or to sort.

{  "provider": {    "only": ["example-provider", "another-provider"],    "ignore": ["another-provider/legacy-pool"]  }}

An entry in ignore wins over the same endpoint in only. Fallbacks stay inside the allowed set.

Sorting

sort replaces the price band and the balanced order with a strict order over all eligible endpoints. Ties are broken by a stable endpoint identifier.

Throughput, latency and quality come from the platform's own measurements of each endpoint, not from provider claims. throughput, latency and quality ignore price; combine them with max_price when cost matters:

curl -X POST "https://api.numes.ai/api/v1/chat/completions" \  -H "Authorization: Bearer $NUMES_API_KEY" \  -H "Content-Type: application/json" \  -d '{    "model": "openai/gpt-6-luna",    "messages": [      {        "role": "user",        "content": "Write a limerick about latency."      }    ],    "provider": {      "sort": "throughput",      "max_price": {        "prompt": 0.000001,        "completion": 0.000004      }    }  }'

Quality sort

sort: "quality" is a Numes extension. It ranks endpoints by how often their responses satisfy the checkable parts of a request: tool calls that name a declared function with arguments matching its JSON Schema, honor tool_choice and are well-formed JSON; json_object and json_schema output that parses and conforms; and a finish reason and output type that match what you asked for.

• Measurements are kept separately for each kind of request, such as chat with tools, chat with response_format, both together, or embeddings. Your request is ranked on measurements for its own kind.

• Responses that can't be checked objectively, such as free-form chat, aren't counted. Your own cancellations and max_tokens cut-offs never count against a provider.

• Quality isn't a judgment of whether an answer is correct, helpful or safe. A well-formed tool call with the wrong city still counts as valid.

• Validation runs on the response in memory. Prompts, output, tool names, arguments and schemas are never stored.

Use it for agents, coding tools and extraction pipelines that break on a malformed tool call or invalid JSON, especially for models served on many engines and quantizations. Skip it for free-form chat: there's nothing to measure, so it adds no signal. Quality sort doesn't affect billing; an invalid response that reached you is billed like any other.

curl -X POST "https://api.numes.ai/api/v1/chat/completions" \  -H "Authorization: Bearer $NUMES_API_KEY" \  -H "Content-Type: application/json" \  -d '{    "model": "openai/gpt-6-luna",    "messages": [      {        "role": "user",        "content": "Extract the invoice number and total from: Invoice 4471, total 98.20 EUR."      }    ],    "response_format": {      "type": "json_schema",      "json_schema": {        "name": "invoice",        "strict": true,        "schema": {          "type": "object",          "properties": {            "invoice_number": {              "type": "string"            },            "total": {              "type": "number"            },            "currency": {              "type": "string"            }          },          "required": [            "invoice_number",            "total",            "currency"          ],          "additionalProperties": false        }      }    },    "provider": {      "sort": "quality",      "max_price": {        "prompt": 0.000001,        "completion": 0.000004      }    }  }'

Each model page has a Task quality chart per endpoint, and the dashboard Models page can sort by Task quality. See Endpoint statistics.

Maximum price

max_price excludes endpoints whose rates exceed your ceilings.

• Values are per token, in your key's currency: US dollars for a USD key, sats for a BTC key.

• Ceilings compare against the endpoint's listed rates.

• The prompt ceiling also applies to a separately priced cache-write rate, and the completion ceiling to a separately priced reasoning rate. Cache reads never cost more than the prompt rate.

• Only prompt and completion are accepted, as non-negative numbers.

• The ceiling applies to every fallback too.

curl -X POST "https://api.numes.ai/api/v1/chat/completions" \  -H "Authorization: Bearer $NUMES_API_KEY" \  -H "Content-Type: application/json" \  -d '{    "model": "openai/gpt-6-luna",    "messages": [      {        "role": "user",        "content": "Hello"      }    ],    "provider": {      "max_price": {        "prompt": 5e-7,        "completion": 0.000002      }    }  }'

Require parameters

Generation hints are forwarded by default, including unfamiliar parameters. With require_parameters: true, supplied parameters and ordinary features must have declared support, or the endpoint is excluded:

• temperature, top_p, top_k, seed, stop

• max_tokens or max_completion_tokens (both count as max_tokens)

• tool_choice, when set to "required" or a named function; "auto" and "none" don't count

Sampling parameters sent as null count as not sent. Client functions, response_format and reasoning become routing requirements only with this flag. Media, hosted tools, explicit cache directives, access and privacy restrictions remain mandatory.

{  "model": "openai/gpt-6-luna",  "messages": [{ "role": "user", "content": "Pick a random number." }],  "temperature": 1.2,  "seed": 42,  "provider": { "require_parameters": true }}

Data collection and retention

Every endpoint declares whether it collects request data, its retention (none, limited, indefinite or unknown) and where it processes requests.

• data_collection: "deny" excludes endpoints that declare collection, and endpoints with no declaration.

• zdr: true requires an endpoint that declares no retention.

Endpoints that resell an upstream API are subject to that upstream's terms. Unless the provider declares otherwise, they count as collecting data with unknown retention, so deny and zdr exclude them.

curl -X POST "https://api.numes.ai/api/v1/chat/completions" \  -H "Authorization: Bearer $NUMES_API_KEY" \  -H "Content-Type: application/json" \  -d '{    "model": "openai/gpt-6-luna",    "messages": [      {        "role": "user",        "content": "Review this contract clause."      }    ],    "provider": {      "data_collection": "deny",      "zdr": true    }  }'

Declarations are the provider's contractual commitments, reviewed at approval. Apply your own diligence for regulated workloads. See Provider data policies.

Quantization

quantizations keeps only endpoints whose declared weight precision is in the list. Values are exact and lowercase:

int4, int8, fp4, mxfp4, nvfp4, fp6, fp8, mxfp8, fp16, bf16, fp32

An endpoint that declares no quantization is excluded whenever the filter is set. The endpoints API shows each endpoint's quantization.

{  "provider": { "quantizations": ["fp8", "fp16", "bf16"] }}

Regions

regions is a Numes extension. It keeps only endpoints that declare processing in at least one of the listed countries or regions. Each value is an ISO country code such as DE or a region identifier the endpoint declares; matching is case-insensitive. An endpoint without a location declaration is excluded whenever the filter is set.

curl -X POST "https://api.numes.ai/api/v1/chat/completions" \  -H "Authorization: Bearer $NUMES_API_KEY" \  -H "Content-Type: application/json" \  -d '{    "model": "openai/gpt-6-luna",    "messages": [      {        "role": "user",        "content": "Hello"      }    ],    "provider": {      "regions": [        "DE",        "FR",        "NL",        "SE",        "FI"      ]    }  }'

OpenRouter offers EU routing through a separate base URL; Numes uses this per-request field instead.

Recipes

See which endpoint served a request

• The response's provider field names the provider.

• Send X-OpenRouter-Metadata: enabled to receive an openrouter_metadata object with provider_slug and endpoint (the tag). On a stream it arrives with the final usage chunk.

• GET /generation returns the serving endpoint and every endpoint tried, including failed fallbacks.

• Logs in the dashboard show the same route for each request.

curl -X POST "https://api.numes.ai/api/v1/chat/completions" \  -H "Authorization: Bearer $NUMES_API_KEY" \  -H "X-OpenRouter-Metadata: enabled" \  -H "Content-Type: application/json" \  -d '{    "model": "openai/gpt-6-luna",    "messages": [      {        "role": "user",        "content": "Hello"      }    ],    "provider": {      "sort": "latency"    }  }'

Troubleshooting

Next steps

Generation hints

Ordinary sampling, reasoning, client function and response-format controls are passed to the upstream by default. Missing catalogue capability declarations do not exclude an endpoint. Set provider.require_parameters: true to opt into strict declared-parameter matching. Access, privacy, cache isolation, media accounting and hosted execution remain admission requirements.