Sticky sessions

A model on Numes is usually served by several endpoints. Sticky routing sends the follow-up requests of a conversation back to the endpoint that served it last, as long as that endpoint is still eligible and among the lowest-priced. Staying on one endpoint makes provider prompt caches more likely to hit, which lowers latency and can lower cost, and keeps provider-specific state, such as signed reasoning, with the provider that produced it.

Stickiness works without any setup: requests of the same conversation are grouped automatically. Send a session ID when you want to control which requests belong together.

After a request succeeds, its endpoint is remembered for that conversation, and the next request tries it first. Records of inactive conversations expire. Stickiness is a preference, not a reservation: it never makes an ineligible endpoint eligible and never bypasses your routing filters, price limits, health checks or capacity.

Set a session ID

Send session_id in the body:

curl -X POST "https://api.numes.ai/api/v1/chat/completions" \  -H "Authorization: Bearer $NUMES_API_KEY" \  -H "Content-Type: application/json" \  -d '{    "model": "openai/gpt-6-luna",    "session_id": "conv_01J9ZK4Q7V6N",    "messages": [      {        "role": "system",        "content": "You are a concise assistant for a hiking app."      },      {        "role": "user",        "content": "Suggest a two-hour loop near a lake."      }    ]  }'

Or send the X-Session-Id header, which is convenient when a proxy or framework owns the request body:

curl -X POST "https://api.numes.ai/api/v1/chat/completions" \  -H "Authorization: Bearer $NUMES_API_KEY" \  -H "X-Session-Id: conv_01J9ZK4Q7V6N" \  -H "Content-Type: application/json" \  -d '{    "model": "openai/gpt-6-luna",    "messages": [      {        "role": "system",        "content": "You are a concise assistant for a hiking app."      },      {        "role": "user",        "content": "Suggest a two-hour loop near a lake."      }    ]  }'

Use an opaque, stable identifier per conversation, such as a UUID or your own conversation ID. Don't put personal data in it.

Precedence

The first source present groups the conversation:

A lower-priority source is ignored when a higher one is present. For example, with both prompt_cache_key and session_id, only prompt_cache_key counts.

Automatic grouping follows the start of the conversation: appending turns keeps the conversation together as long as its opening messages stay the same. Two conversations that open identically are grouped together. That's harmless, since stickiness never affects what the model sees, but send a session_id if you want them routed independently.

When stickiness applies

A follow-up sent at the same time as, or immediately after, the previous request may be routed before its endpoint is recorded. Send the turns of a conversation one after another when stickiness matters.

Scope and privacy

• Scoped to your account and the model. The same session ID under another account, or with another model, is a different session. All keys of the same account share sessions.

• Your value stays private. Your session_id, header value or prompt_cache_key isn't stored or sent to providers, and GET /generation doesn't return it.

• What providers can receive. Endpoints that accept a session or cache key receive an opaque hint with no content or account identifiers, so their own cache routing can benefit.

Examples

Reuse one ID for every turn of a conversation. With the OpenAI SDK, send session_id as an extra body field, or send the header per request:

import osimport uuidfrom openai import OpenAI​client = OpenAI(base_url="https://api.numes.ai/api/v1", api_key=os.environ["NUMES_API_KEY"])​session_id = f"conv_{uuid.uuid4().hex}"messages = [{"role": "system", "content": "You are a concise assistant for a hiking app."}]​for question in ["Suggest a two-hour loop near a lake.", "Is it suitable for children?"]:    messages.append({"role": "user", "content": question})    response = client.chat.completions.create(        model="openai/gpt-6-luna",        messages=messages,        extra_body={"session_id": session_id},        # or: extra_headers={"X-Session-Id": session_id},    )    answer = response.choices[0].message.content    messages.append({"role": "assistant", "content": answer})    print(getattr(response, "provider", None), answer)

The response's provider field shows which provider served each turn. To see the exact endpoint, send X-OpenRouter-Metadata: true and read openrouter_metadata.endpoint.

Limits of stickiness

• No cache guarantee. Returning to the same endpoint makes a provider cache hit more likely; whether the engine still holds the prefix, and whether it caches at all, is up to the provider.

• No conversation storage. Numes does not store your messages between requests. Send the full conversation on every turn.

• No capacity reservation. A busy sticky endpoint is skipped rather than waited for.

Best practices

• One ID per conversation. Reusing one ID across unrelated conversations concentrates them on one endpoint without any cache benefit.

• Keep the opening stable. Changing the system prompt changes the implicit key and the cacheable prefix.

• Leave order and sort unset for conversations where cache reuse matters, and use filters such as only or max_price instead.

• Use session_id for affinity, prompt_cache_key to add a cache hint. Neither one narrows routing.

Next steps