# Provider routing

> How Numes chooses a provider endpoint for each request, and how the provider object lets you sort, pin, filter and cap prices.

Source: https://0.0.0.0:3000/docs/routing/provider-routing
Documentation index: https://0.0.0.0:3000/docs/llms.txt

A model on Numes is usually served by several independent provider endpoints, each with its own price, speed, location and data policy. For every request, routing decides which endpoints are eligible, orders them and tries them in turn until one delivers output. The `provider` object changes that behavior.

## The provider object

Add `provider` to a chat completion, embedding or image generation request. Every field is optional; unknown fields are rejected with `400 invalid_request`. Speech and transcription requests use default routing.

```json
{
  "model": "openai/gpt-6-luna",
  "messages": [{ "role": "user", "content": "Hello" }],
  "provider": {
    "sort": "throughput",
    "ignore": ["example-provider"],
    "max_price": { "prompt": 0.000001, "completion": 0.000004 }
  }
}
```

:::fields provider
- `order` `string[]` — Endpoints to try first, in this order. Each entry is a provider slug or a `provider/endpoint` tag. At most 100 entries.
- `allow_fallbacks` `boolean` default `true` — When `false`, the request makes exactly one try, and unlisted endpoints are dropped if `order` is set.
- `only` `string[]` — Allow list. Only endpoints matching an entry are eligible. At most 100 entries.
- `ignore` `string[]` — Deny list. Endpoints matching an entry are never used. At most 100 entries.
- `sort` `string` — `price`, `throughput`, `latency` or `quality`. Replaces the default order with a strict one.
- `max_price` `object` — Price ceilings per token, in your key's currency.
  - `prompt` `number` — Highest acceptable prompt rate. Also applies to a separate cache-write rate.
  - `completion` `number` — Highest acceptable completion rate. Also applies to a separate reasoning rate.
- `require_parameters` `boolean` default `false` — Only use endpoints that declare every sampling parameter you sent, and `tool_choice` when you force a tool call.
- `data_collection` `string` — `allow` or `deny`. `deny` excludes endpoints that collect data or don't declare otherwise.
- `zdr` `boolean` — When `true`, only endpoints that declare no retention are eligible.
- `quantizations` `string[]` — Only endpoints whose declared weight precision is in the list. At most 100 entries.
- `regions` `string[]` — Only endpoints that declare processing in one of these countries or regions. A Numes extension. At most 64 unique values.
:::

List entries must be 1 to 256 characters. Object-form `sort`, percentile thresholds and other OpenRouter routing fields not listed here aren't supported.

## Default routing

With no `provider` object, each request goes through three stages.

### 1. Eligibility

An endpoint is a candidate only when every one of these passes. They're checked for each request, and again before each fallback:

| Check | Passes when |
| --- | --- |
| Model | The endpoint serves the requested model |
| Approval and verification | The provider, endpoint and offer are approved, not paused or suspended, and pass [verification](/docs/routing/verification) |
| Health | The endpoint is connected and ready, and the model isn't failing repeatedly on it |
| Capacity | The endpoint has room for the request |
| Currency | The offer is priced in your key's currency |
| Context | Your prompt and `max_tokens` fit the endpoint's context and output limits |
| Capabilities | The endpoint supports every modality and feature the request uses: tools, JSON output, reasoning controls, cache directives |
| Your filters | Every `provider` filter you set, and any account policy |

If no endpoint passes, the request fails with `503` and `error.metadata.error_type` `provider_unavailable`. An approved offer keeps serving while a replacement revision or price change is under review.

### 2. Price band

Routing estimates what this specific request would cost on each eligible endpoint, from your prompt, `max_tokens` and any per-request or media charges. An endpoint with cheap prompts and expensive completions can therefore rank differently for a long prompt than for a long answer. Endpoints whose estimate is close to the lowest form the **price band**.

This estimate is separate from the hold on your balance; see [Credits](/docs/account/credits).

### 3. Order within the band

Inside the band, routing balances price, reliability and performance across the eligible endpoints, with randomization to spread load. New offers receive some traffic so they can build a track record. Endpoints outside the band follow in ascending order of estimated price.

Routing tries the first endpoint. If it fails before any output, the remaining endpoints are re-checked and the next one is tried, up to a limited number of tries. See [Fallbacks and reliability](/docs/routing/fallbacks).

### Session stickiness

Consecutive requests from the same conversation go to the same endpoint while it's still eligible, which improves prompt-cache hits. Stickiness is keyed on `prompt_cache_key`, then `session_id`, then the `X-Session-Id` header. It never overrides eligibility and applies only within the price band, and it's off whenever you set `order` or `sort`. See [Sessions](/docs/features/sessions).

## Selectors

`order`, `only` and `ignore` accept two kinds of selector:

| Selector | Example | Matches |
| --- | --- | --- |
| Provider slug | `example-provider` | Every endpoint of that provider |
| Endpoint tag | `example-provider/eu-west` | One endpoint |

Selectors are stable routing slugs that don't change when a provider renames itself. Matching is case-insensitive. Display names such as `Example Provider` don't match.

Each endpoint's `tag` in the endpoints API is its selector; the part before the slash is the provider slug:

```request
{
  "method": "GET",
  "path": "/models/openai/gpt-6-luna/endpoints"
}
```

The same values appear as `provider_slug` and `endpoint` in [generation metadata](/docs/api-reference/chat/get-generation), so you can copy the selector of an endpoint that served you well.

## Order and fallbacks

`order` moves the listed endpoints to the front, in list order. A provider slug ranks all of that provider's endpoints at its position. Unlisted endpoints still follow as fallbacks.

```request
{
  "method": "POST",
  "path": "/chat/completions",
  "title": "Prefer two providers, then anything else",
  "body": {
    "model": "openai/gpt-6-luna",
    "messages": [{ "role": "user", "content": "Summarize the plot of Hamlet in two sentences." }],
    "provider": {
      "order": ["example-provider", "another-provider/fast-gpu"]
    }
  }
}
```

Setting `order` turns off the price band, the balanced order and session stickiness. Unlisted endpoints follow in a fixed order. Add `sort` to order endpoints that share a position, such as several endpoints of a listed provider, and the unlisted remainder. A sort never moves an endpoint ahead of one listed earlier.

```json
{
  "provider": {
    "order": ["example-provider"],
    "sort": "price"
  }
}
```

`allow_fallbacks: false` limits the request to one try. With `order`, unlisted endpoints are also dropped, so that try goes to the first listed endpoint that's eligible. If none is eligible, the request fails with `503`; if it fails before output, there's no second try. See [Turn fallbacks off](/docs/routing/fallbacks#turn-fallbacks-off).

## Allow and deny lists

`only` restricts routing to matching endpoints; `ignore` removes matching endpoints. Both use the same selectors and leave ordering to the default or to `sort`.

```json
{
  "provider": {
    "only": ["example-provider", "another-provider"],
    "ignore": ["another-provider/legacy-pool"]
  }
}
```

An entry in `ignore` wins over the same endpoint in `only`. Fallbacks stay inside the allowed set.

## Sorting

`sort` replaces the price band and the balanced order with a strict order over all eligible endpoints. Ties are broken by a stable endpoint identifier.

| Value | Orders by | Endpoints without measurements |
| --- | --- | --- |
| `price` | Estimated cost of this request, lowest first | Not applicable |
| `throughput` | Fastest observed output first | Last |
| `latency` | Fastest observed time to first token first | Last |
| `quality` | Highest measured quality for this kind of request first | Last, with occasional exploration |

Throughput, latency and quality come from the platform's own measurements of each endpoint, not from provider claims. `throughput`, `latency` and `quality` ignore price; combine them with `max_price` when cost matters:

```request
{
  "method": "POST",
  "path": "/chat/completions",
  "title": "Fastest endpoint under a price ceiling",
  "body": {
    "model": "openai/gpt-6-luna",
    "messages": [{ "role": "user", "content": "Write a limerick about latency." }],
    "provider": {
      "sort": "throughput",
      "max_price": { "prompt": 0.000001, "completion": 0.000004 }
    }
  }
}
```

### Quality sort

`sort: "quality"` is a Numes extension. It ranks endpoints by how often their responses satisfy the checkable parts of a request: tool calls that name a declared function with arguments matching its JSON Schema, honor `tool_choice` and are well-formed JSON; `json_object` and `json_schema` output that parses and conforms; and a finish reason and output type that match what you asked for.

- Measurements are kept separately for each kind of request, such as chat with tools, chat with `response_format`, both together, or embeddings. Your request is ranked on measurements for its own kind.
- Responses that can't be checked objectively, such as free-form chat, aren't counted. Your own cancellations and `max_tokens` cut-offs never count against a provider.
- Quality isn't a judgment of whether an answer is correct, helpful or safe. A well-formed tool call with the wrong city still counts as valid.
- Validation runs on the response in memory. Prompts, output, tool names, arguments and schemas are never stored.

Use it for agents, coding tools and extraction pipelines that break on a malformed tool call or invalid JSON, especially for models served on many engines and quantizations. Skip it for free-form chat: there's nothing to measure, so it adds no signal. Quality sort doesn't affect billing; an invalid response that reached you is billed like any other.

```request
{
  "method": "POST",
  "path": "/chat/completions",
  "title": "Valid structured output within a price ceiling",
  "body": {
    "model": "openai/gpt-6-luna",
    "messages": [{ "role": "user", "content": "Extract the invoice number and total from: Invoice 4471, total 98.20 EUR." }],
    "response_format": {
      "type": "json_schema",
      "json_schema": {
        "name": "invoice",
        "strict": true,
        "schema": {
          "type": "object",
          "properties": {
            "invoice_number": { "type": "string" },
            "total": { "type": "number" },
            "currency": { "type": "string" }
          },
          "required": ["invoice_number", "total", "currency"],
          "additionalProperties": false
        }
      }
    },
    "provider": {
      "sort": "quality",
      "max_price": { "prompt": 0.000001, "completion": 0.000004 }
    }
  }
}
```

Each model page has a **Task quality** chart per endpoint, and the dashboard **Models** page can sort by **Task quality**. See [Endpoint statistics](/docs/models#endpoint-statistics).

## Maximum price

`max_price` excludes endpoints whose rates exceed your ceilings.

- Values are **per token**, in your key's currency: US dollars for a USD key, sats for a BTC key.
- Ceilings compare against the endpoint's listed rates.
- The `prompt` ceiling also applies to a separately priced cache-write rate, and the `completion` ceiling to a separately priced reasoning rate. Cache reads never cost more than the prompt rate.
- Only `prompt` and `completion` are accepted, as non-negative numbers.
- The ceiling applies to every fallback too.

:::warning Units differ from OpenRouter
OpenRouter's `max_price` is per million tokens. Here it's per token: $1 per million prompt tokens is `"prompt": 0.000001` for a USD key, and 500 sats per million completion tokens is `"completion": 0.0005` for a BTC key.
:::

```request
{
  "method": "POST",
  "path": "/chat/completions",
  "title": "Cap prices",
  "body": {
    "model": "openai/gpt-6-luna",
    "messages": [{ "role": "user", "content": "Hello" }],
    "provider": {
      "max_price": { "prompt": 0.0000005, "completion": 0.000002 }
    }
  }
}
```

## Require parameters {#require-parameters}

Generation hints are forwarded by default, including unfamiliar parameters. With `require_parameters: true`, supplied parameters and ordinary features must have declared support, or the endpoint is excluded:

- `temperature`, `top_p`, `top_k`, `seed`, `stop`
- `max_tokens` or `max_completion_tokens` (both count as `max_tokens`)
- `tool_choice`, when set to `"required"` or a named function; `"auto"` and `"none"` don't count

Sampling parameters sent as `null` count as not sent. Client functions, `response_format` and reasoning become routing requirements only with this flag. Media, hosted tools, explicit cache directives, access and privacy restrictions remain mandatory.

```json
{
  "model": "openai/gpt-6-luna",
  "messages": [{ "role": "user", "content": "Pick a random number." }],
  "temperature": 1.2,
  "seed": 42,
  "provider": { "require_parameters": true }
}
```

## Data collection and retention

Every endpoint declares whether it collects request data, its retention (`none`, `limited`, `indefinite` or `unknown`) and where it processes requests.

- `data_collection: "deny"` excludes endpoints that declare collection, and endpoints with no declaration.
- `zdr: true` requires an endpoint that declares no retention.

Endpoints that resell an upstream API are subject to that upstream's terms. Unless the provider declares otherwise, they count as collecting data with unknown retention, so `deny` and `zdr` exclude them.

```request
{
  "method": "POST",
  "path": "/chat/completions",
  "title": "No data collection, no retention",
  "body": {
    "model": "openai/gpt-6-luna",
    "messages": [{ "role": "user", "content": "Review this contract clause." }],
    "provider": { "data_collection": "deny", "zdr": true }
  }
}
```

Declarations are the provider's contractual commitments, reviewed at approval. Apply your own diligence for regulated workloads. See [Provider data policies](/docs/privacy/provider-data-policies).

## Quantization

`quantizations` keeps only endpoints whose declared weight precision is in the list. Values are exact and lowercase:

`int4`, `int8`, `fp4`, `mxfp4`, `nvfp4`, `fp6`, `fp8`, `mxfp8`, `fp16`, `bf16`, `fp32`

An endpoint that declares no quantization is excluded whenever the filter is set. The endpoints API shows each endpoint's `quantization`.

```json
{
  "provider": { "quantizations": ["fp8", "fp16", "bf16"] }
}
```

## Regions

`regions` is a Numes extension. It keeps only endpoints that declare processing in at least one of the listed countries or regions. Each value is an ISO country code such as `DE` or a region identifier the endpoint declares; matching is case-insensitive. An endpoint without a location declaration is excluded whenever the filter is set.

```request
{
  "method": "POST",
  "path": "/chat/completions",
  "title": "Keep processing in selected EU countries",
  "body": {
    "model": "openai/gpt-6-luna",
    "messages": [{ "role": "user", "content": "Hello" }],
    "provider": { "regions": ["DE", "FR", "NL", "SE", "FI"] }
  }
}
```

OpenRouter offers EU routing through a separate base URL; Numes uses this per-request field instead.

## Recipes

| Goal | `provider` |
| --- | --- |
| Lowest price, deterministic | `{ "sort": "price" }` |
| Highest throughput | `{ "sort": "throughput" }` |
| Lowest time to first token | `{ "sort": "latency" }` |
| Most reliable tool calls and JSON | `{ "sort": "quality" }` |
| One exact endpoint, never another | `{ "order": ["provider/endpoint"], "allow_fallbacks": false }` |
| Prefer a provider, then cheapest | `{ "order": ["provider"], "sort": "price" }` |
| Exclude a provider | `{ "ignore": ["provider"] }` |
| No data collection or retention | `{ "data_collection": "deny", "zdr": true }` |
| Cap spend per token | `{ "max_price": { "prompt": 0.000001, "completion": 0.000004 } }` |
| Full-precision weights only | `{ "quantizations": ["fp16", "bf16", "fp32"] }` |

## See which endpoint served a request

- The response's `provider` field names the provider.
- Send `X-OpenRouter-Metadata: enabled` to receive an `openrouter_metadata` object with `provider_slug` and `endpoint` (the tag). On a stream it arrives with the final usage chunk.
- [`GET /generation`](/docs/api-reference/chat/get-generation) returns the serving endpoint and every endpoint tried, including failed fallbacks.
- [Logs](/dashboard/logs) in the dashboard show the same route for each request.

```request
{
  "method": "POST",
  "path": "/chat/completions",
  "headers": { "X-OpenRouter-Metadata": "enabled" },
  "body": {
    "model": "openai/gpt-6-luna",
    "messages": [{ "role": "user", "content": "Hello" }],
    "provider": { "sort": "latency" }
  }
}
```

## Troubleshooting

| Response | Likely cause | What to do |
| --- | --- | --- |
| `503 provider_unavailable` | Your filters excluded every endpoint, no endpoint supports a feature you used, your prompt plus `max_tokens` exceeds every context window, or every endpoint is unhealthy or full | Remove filters one at a time, lower `max_tokens`, and compare against `GET /models/{author}/{slug}/endpoints` |
| `503 provider_overloaded` | The selected endpoints ran out of capacity | Retry with backoff, or widen `only`, `regions` or `max_price` |
| `400 invalid_request` | An unknown field in `provider`, such as an object-form `sort` or `max_price.request`; a wrong type; a value outside the documented set; a list or entry over its limit; or duplicate or malformed `regions` | Use only the fields, values and lengths on this page |
| `order` or `only` has no effect | The selector doesn't match a slug or tag; display names never match | Copy `tag` from the endpoints API |
| `require_parameters` returns `503` | No endpoint declares one of the sampling parameters you sent, or `tool_choice` for a forced tool call | Check each endpoint's `supported_parameters`, or drop the parameter |

## Next steps

:::cards
- {branch} [Fallbacks and reliability](/docs/routing/fallbacks) Health, automatic failover and client-side model fallback.
- {shield-check} [Verification and trust](/docs/routing/verification) What makes an endpoint eligible.
- {lock} [Provider data policies](/docs/privacy/provider-data-policies) Declarations behind `data_collection`, `zdr` and `regions`.
- {globe} [Models](/docs/models) Find models, endpoints and their statistics.
:::

## Generation hints

Ordinary sampling, reasoning, client function and response-format controls are
passed to the upstream by default. Missing catalogue capability declarations do
not exclude an endpoint. Set `provider.require_parameters: true` to opt into
strict declared-parameter matching. Access, privacy, cache isolation, media
accounting and hosted execution remain admission requirements.