# Sticky sessions

> Keep the requests of one conversation on the same endpoint with session_id or X-Session-Id, so provider caches and reasoning state can be reused.

Source: https://0.0.0.0:3000/docs/features/sessions
Documentation index: https://0.0.0.0:3000/docs/llms.txt

A model on Numes is usually served by several endpoints. Sticky routing sends the follow-up requests of a conversation back to the endpoint that served it last, as long as that endpoint is still eligible and among the lowest-priced. Staying on one endpoint makes provider prompt caches more likely to hit, which lowers latency and can lower cost, and keeps provider-specific state, such as signed reasoning, with the provider that produced it.

Stickiness works without any setup: requests of the same conversation are grouped automatically. Send a session ID when you want to control which requests belong together.

After a request succeeds, its endpoint is remembered for that conversation, and the next request tries it first. Records of inactive conversations expire. Stickiness is a preference, not a reservation: it never makes an ineligible endpoint eligible and never bypasses your routing filters, price limits, health checks or capacity.

## Set a session ID

Send `session_id` in the body:

```request
{
  "method": "POST",
  "path": "/chat/completions",
  "body": {
    "model": "openai/gpt-6-luna",
    "session_id": "conv_01J9ZK4Q7V6N",
    "messages": [
      { "role": "system", "content": "You are a concise assistant for a hiking app." },
      { "role": "user", "content": "Suggest a two-hour loop near a lake." }
    ]
  }
}
```

Or send the `X-Session-Id` header, which is convenient when a proxy or framework owns the request body:

```request
{
  "method": "POST",
  "path": "/chat/completions",
  "headers": { "X-Session-Id": "conv_01J9ZK4Q7V6N" },
  "body": {
    "model": "openai/gpt-6-luna",
    "messages": [
      { "role": "system", "content": "You are a concise assistant for a hiking app." },
      { "role": "user", "content": "Suggest a two-hour loop near a lake." }
    ]
  }
}
```

:::fields Session fields
- `session_id` `string` — Body field. 1–256 bytes, not blank, no control characters. Wins over the header.
- `X-Session-Id` `string` — Request header with the same format. Used when the body has no `session_id`.
:::

Use an opaque, stable identifier per conversation, such as a UUID or your own conversation ID. Don't put personal data in it.

## Precedence

The first source present groups the conversation:

| Priority | Source | Notes |
| --- | --- | --- |
| 1 | `prompt_cache_key` | Also sends a cache hint to endpoints that accept one. See [Prompt caching](/docs/features/prompt-caching). |
| 2 | `session_id` in the body | |
| 3 | `X-Session-Id` header | |
| 4 | Automatic | Used when none of the above is sent. |

A lower-priority source is ignored when a higher one is present. For example, with both `prompt_cache_key` and `session_id`, only `prompt_cache_key` counts.

Automatic grouping follows the start of the conversation: appending turns keeps the conversation together as long as its opening messages stay the same. Two conversations that open identically are grouped together. That's harmless, since stickiness never affects what the model sees, but send a `session_id` if you want them routed independently.

## When stickiness applies

| Situation | Behavior |
| --- | --- |
| No `provider.order` and no `provider.sort` | Stickiness applies. |
| `provider.order` or `provider.sort` is set | Stickiness is off; your explicit order or sort decides. |
| `provider.only`, `ignore`, `max_price` or other filters | Stickiness applies among the endpoints that pass the filters. |
| The sticky endpoint is ineligible, unhealthy or full | It is skipped and selection proceeds normally. |
| The sticky endpoint is no longer among the lowest-priced endpoints | It is skipped. A conversation that moved to a pricier endpoint because cheaper ones were full returns to the cheaper ones as soon as they can serve it again. |
| The sticky endpoint fails before output | Normal fallback applies. If another endpoint then succeeds, the conversation moves to it. |
| The conversation has been inactive for a while | The record has expired; selection starts fresh. |

A follow-up sent at the same time as, or immediately after, the previous request may be routed before its endpoint is recorded. Send the turns of a conversation one after another when stickiness matters.

## Scope and privacy

- **Scoped to your account and the model.** The same session ID under another account, or with another model, is a different session. All keys of the same account share sessions.
- **Your value stays private.** Your `session_id`, header value or `prompt_cache_key` isn't stored or sent to providers, and [`GET /generation`](/docs/api-reference/chat/get-generation) doesn't return it.
- **What providers can receive.** Endpoints that accept a session or cache key receive an opaque hint with no content or account identifiers, so their own cache routing can benefit.

## Examples

Reuse one ID for every turn of a conversation. With the OpenAI SDK, send `session_id` as an extra body field, or send the header per request:

:::code-group
```python title="Python"
import os
import uuid
from openai import OpenAI

client = OpenAI(base_url="https://api.numes.ai/api/v1", api_key=os.environ["NUMES_API_KEY"])

session_id = f"conv_{uuid.uuid4().hex}"
messages = [{"role": "system", "content": "You are a concise assistant for a hiking app."}]

for question in ["Suggest a two-hour loop near a lake.", "Is it suitable for children?"]:
    messages.append({"role": "user", "content": question})
    response = client.chat.completions.create(
        model="openai/gpt-6-luna",
        messages=messages,
        extra_body={"session_id": session_id},
        # or: extra_headers={"X-Session-Id": session_id},
    )
    answer = response.choices[0].message.content
    messages.append({"role": "assistant", "content": answer})
    print(getattr(response, "provider", None), answer)
```

```typescript title="TypeScript"
import OpenAI from "openai";
import { randomUUID } from "node:crypto";
import type { ChatCompletionMessageParam } from "openai/resources/chat/completions";

const client = new OpenAI({ baseURL: "https://api.numes.ai/api/v1", apiKey: process.env.NUMES_API_KEY });

const sessionId = `conv_${randomUUID()}`;
const messages: ChatCompletionMessageParam[] = [
  { role: "system", content: "You are a concise assistant for a hiking app." },
];

for (const question of ["Suggest a two-hour loop near a lake.", "Is it suitable for children?"]) {
  messages.push({ role: "user", content: question });
  const response = await client.chat.completions.create(
    { model: "openai/gpt-6-luna", messages },
    { headers: { "X-Session-Id": sessionId } },
  );
  const answer = response.choices[0].message.content ?? "";
  messages.push({ role: "assistant", content: answer });
  console.log((response as { provider?: string }).provider, answer);
}
```
:::

The response's `provider` field shows which provider served each turn. To see the exact endpoint, send `X-OpenRouter-Metadata: true` and read `openrouter_metadata.endpoint`.

## Limits of stickiness

- **No cache guarantee.** Returning to the same endpoint makes a provider cache hit more likely; whether the engine still holds the prefix, and whether it caches at all, is up to the provider.
- **No conversation storage.** Numes does not store your messages between requests. Send the full conversation on every turn.
- **No capacity reservation.** A busy sticky endpoint is skipped rather than waited for.

## Best practices

- **One ID per conversation.** Reusing one ID across unrelated conversations concentrates them on one endpoint without any cache benefit.
- **Keep the opening stable.** Changing the system prompt changes the implicit key and the cacheable prefix.
- **Leave `order` and `sort` unset** for conversations where cache reuse matters, and use filters such as `only` or `max_price` instead.
- **Use `session_id` for affinity, `prompt_cache_key` to add a cache hint.** Neither one narrows routing.

## Next steps

:::cards
- {database} [Prompt caching](/docs/features/prompt-caching) Cache keys, directives, usage and pricing.
- {network} [Provider routing](/docs/routing/provider-routing) Order, sort and filters, and how they interact with stickiness.
- {route} [Fallbacks](/docs/routing/fallbacks) What happens when an endpoint fails.
- {brain} [Reasoning](/docs/features/reasoning) Carry reasoning state through multi-turn conversations.
:::