# Limits

> Rate limits, body size, media and structure limits, time limits, and the balance and spending limits a request must fit.

Source: https://0.0.0.0:3000/docs/api-reference/limits
Documentation index: https://0.0.0.0:3000/docs/llms.txt

Every request must fit several independent limits before it reaches a provider. This page lists each limit, the error it produces, and where to check your own values. Values may change.

| Limit | Error when exceeded | How to check |
| --- | --- | --- |
| [Rate limits](#rate-limits) | `429 rate_limited` with `Retry-After` | `Retry-After` header |
| [Body size](#request-body-size) | `413 payload_too_large` | This page |
| [Media inputs](#media-inputs) | `400 invalid_request` | This page |
| [Structure](#structural-limits) | `400 invalid_request` | This page |
| [Request duration](#timeouts) | `408 timeout`, or an in-band stream error | This page |
| [Funds](#credits-and-reservations) | `402 insufficient_credits` | [Billing](/dashboard/billing) |
| [Key spending limit](#key-spending-limits) | `402 insufficient_credits` | `GET /key` |
| [Account spending limit](#account-spending-limits) | `402 insufficient_credits` | [Billing](/dashboard/billing) |
| Provider capacity | `503 provider_overloaded` | Retry with backoff |

## Rate limits

Inference requests are rate-limited per key, per account and per key creator, and a request must fit all three. Each scope limits both the number of requests and their token demand: your prompt, plus the full `max_completion_tokens` you set, plus media inputs. A large output limit therefore uses up your allowance faster. Short bursts are absorbed; sustained traffic is held to a steady rate.

When a limit is reached, the response is `429 rate_limited` with the message `rate limit reached` and a `Retry-After` header in whole seconds. Wait that long before the next request with the same key. Renaming a key or changing its spending limit doesn't reset its rate limits. A rate-limited request doesn't use up its `X-Request-Id`.

## Request body size

| Route | Maximum body |
| --- | --- |
| `POST /chat/completions`, `POST /e2ee/chat/completions` | 10 MiB |
| `POST /embeddings` | 10 MiB |
| `POST /images` | 10 MiB |
| `POST /audio/speech` | 1 MiB |
| `POST /audio/transcriptions` | 25 MiB |
| `POST /keys`, `PATCH /keys/{hash}` | 32 KiB |
| `POST /e2ee/resolve` | 4 KiB |
| All `GET` and `DELETE` routes | No body |

A larger body returns `413 payload_too_large`, whether its size is declared in `Content-Length` or it's sent chunked.

## Media inputs

These limits apply to images, PDFs and audio in chat messages, embedding inputs, image references and transcription uploads:

| Limit | Value |
| --- | --- |
| One media item (URL, `data:` URL or upload) | 8 MiB |
| All media in one request | 24 MiB |
| Input audio per request | 1 hour |
| Sources | Publicly reachable HTTPS URLs, or `data:` URLs |
| Types | PNG, JPEG, GIF, WebP, PDF, WAV, MP3, FLAC, M4A, Ogg, WebM, AAC, AIFF, PCM |

A media item that breaks one of these rules, can't be fetched promptly, or points to a non-public destination returns `400 invalid_request`. `data:` URLs also count toward the route's body limit, and base64 adds about a third, so a 10 MiB body carries about 7.5 MiB of inline media. Use HTTPS URLs for more. A transcription upload must fit both the per-item limit and the route's body limit.

## Structural limits

| Field | Limit |
| --- | --- |
| Chat messages and content parts | Aggregate request bytes and JSON structure budget |
| JSON structure | 64 levels, 262144 nodes; normalized transport must also fit its framing budget |
| Chat text, history calls, function names and stop hints | Aggregate request limit; provider semantics |


| Hosted tool `configuration` | 16 settings, 32 KiB |
| `max_tool_calls` | 1 to 128 |
| `max_completion_tokens`, `max_tokens` | 1 to 131072 |

| `session_id`, `X-Session-Id`, `prompt_cache_key`, `user` | 256 characters |
| `X-Request-Id` | 128 characters from `A-Z a-z 0-9 . _ -` |
| `provider.order`, `only`, `ignore`, `quantizations` | 100 entries of up to 256 characters |
| `provider.regions` | 64 unique entries |
| Embeddings `input` | 2048 items; each string up to 4 MiB |
| Embeddings `dimensions` | 1 to 65536 |
| Images `n` | 1 to 10 |
| Images `input_references` | 16 |
| Images `prompt` | 1 MiB |
| Images `size` | `auto` or `WxH`, each side 64 to 8192 pixels |
| Speech `input` | 4096 bytes |
| Transcription multipart form | 32 parts, one `file`, other fields up to 8192 bytes |

The number of `tools` and the size of a function's `description` and `parameters` have no limit of their own; they count toward the body limit. Values outside a range return `400`; they aren't clamped. A model's maximum output is in `top_provider.max_completion_tokens` from [`GET /models`](/docs/api-reference/models/list-models).

## Timeouts

| Scope | Limit | On expiry |
| --- | --- | --- |
| Chat, images and speech, including the whole stream | 10 minutes | `408 timeout` before output; an in-band `timeout` error after output starts |
| Embeddings and transcriptions | 10 minutes | `408 timeout` |
| `GET` routes, key management and `POST /e2ee/resolve` | 10 seconds | The request fails; retry with backoff |

Streams send no keep-alive frames, so set your client's read timeout to at least 10 minutes.

## Credits and reservations

Before a request runs, an upper bound of its cost is held against your balance and spending limits; unused funds are released when the request completes. For token-priced models the hold covers the endpoint's full context and output limits, so a smaller `max_completion_tokens` doesn't reduce it. The request runs only if the hold fits your available balance, which is your balance minus holds for other running requests. Many concurrent requests can use up the available balance through holds alone, even when the balance covers their actual cost.

Balances are per currency: a USD key draws only on the USD balance, and a BTC key only on the BTC balance. When the hold doesn't fit, the response is `402 insufficient_credits`. [`GET /generation`](/docs/api-reference/chat/get-generation) is the authoritative record of what you were charged. See [Credits and payments](/docs/account/credits).

## Key spending limits

An inference key can carry a spending `limit` in its currency, with an optional `limit_reset`:

| `limit_reset` | Period |
| --- | --- |
| `daily` | Resets at 00:00 UTC every day |
| `weekly` | Resets at 00:00 UTC every Monday |
| `monthly` | Resets at 00:00 UTC on the 1st of each month |
| `null` | Lifetime limit, never resets |

A request runs only if the key's spend in the current period, plus holds for its running requests, plus this request's hold stays within the limit. Set the limit in [Keys](/dashboard/keys) or with [`PATCH /keys/{hash}`](/docs/api-reference/keys/update-key). [`GET /key`](/docs/api-reference/keys/get-current-key) returns the calling key's `limit`, `limit_reset` and `limit_remaining`; `limit_remaining` doesn't subtract holds for running requests. See [API keys](/docs/account/api-keys#spending-limits).

## Account spending limits

Owners and administrators can set one spending limit per currency for the whole account in [Billing](/dashboard/billing). It uses the same periods and rule as key limits, applied to all of the account's keys together.

## Next steps

:::cards
- {coins} [Credits and payments](/docs/account/credits) Add funds, auto top-up and balances per currency.
- {key} [API keys](/docs/account/api-keys) Spending limits and reset periods per key.
- {alert} [Errors and debugging](/docs/api-reference/errors) Every error type and the retry strategy.
- {braces} [Parameters](/docs/api-reference/parameters) Field ranges and how they affect routing.
:::