# POST /v1/chat/completions

The OpenAI-compatible chat endpoint. Point any OpenAI SDK or client at `https://demo-api.routerplus.com/v1` with your RouterPlus key, and call any chat model in the catalog, Claude models included. The gateway translates requests, streams and errors between provider dialects.

Image models answer on [POST /v1/images/generations](/docs/api-images) only. The Anthropic-native endpoint is [POST /v1/messages](/docs/api-messages).

```
POST https://demo-api.routerplus.com/v1/chat/completions
```

## Examples

```bash
curl -N https://demo-api.routerplus.com/v1/chat/completions \
  -H "Authorization: Bearer $TM_API_KEY" \
  -H "content-type: application/json" \
  -d '{"model":"claude-haiku-4-5","stream":true,"max_tokens":100,"messages":[{"role":"user","content":"Say hello."}]}'
```

Python:

```python
import os
from openai import OpenAI

client = OpenAI(base_url="https://demo-api.routerplus.com/v1", api_key=os.environ["TM_API_KEY"])

resp = client.chat.completions.create(
    model="claude-sonnet-4-5",  # a Claude model over the OpenAI wire: the gateway translates
    max_tokens=200,
    messages=[{"role": "user", "content": "Say hello in one sentence."}],
)
print(resp.choices[0].message.content)
print(resp.usage)  # prompt/completion tokens, cache splits, cost (USD)
```

TypeScript:

```typescript
import OpenAI from "openai";

const client = new OpenAI({
  baseURL: "https://demo-api.routerplus.com/v1",
  apiKey: process.env.TM_API_KEY,
});

const stream = await client.chat.completions.create({
  model: "gpt-4o-mini",
  stream: true,
  messages: [{ role: "user", content: "Count to five." }],
});

for await (const chunk of stream) {
  process.stdout.write(chunk.choices[0]?.delta?.content ?? "");
  if (chunk.usage) console.log("\n", chunk.usage); // final chunk: tokens, cache splits, cost
}
```

The Python and TypeScript examples use the official `openai` SDK with only the base URL changed.

## Authentication

| Header | Format |
|---|---|
| `Authorization` | `Bearer tm_vk_...` (the standard OpenAI header) |
| `x-api-key` | `tm_vk_...` (also accepted, same key) |

A missing or invalid key returns 401 with `error_type: "auth"`. To identify your app, send `HTTP-Referer` and `X-Title`. They carry no content.

## Request

The body is JSON, at most 10 MB (413 `request_too_large` above that).

| Parameter | Type | Behavior |
|---|---|---|
| `model` | string, required | A catalog id. See [Model discovery](#model-discovery). An unknown id is a 404. The gateway never substitutes another model. Each Claude model also answers to one `rp-` name, for clients such as Cursor that do not send `claude-` names to another base URL: `rp-sonnet-5-5` is `claude-sonnet-5-5` (see [Models & catalog](/docs/models#claude-names-for-cursor)). This route also serves a deployed endpoint from [Optimize](/docs/model-search) (`tm/<name>-v<n>`), with a key of the organization that owns it, and a [dedicated endpoint](/docs/dedicated-endpoints) (`<your-namespace>/<name>`). |
| `messages` | array, required | Roles `system`, `developer`, `user`, `assistant`, `tool`. Content is a string or `text` parts. A model that takes them also accepts `image_url`, `file`, `input_audio` and `video_url` parts (see `architecture.input_modalities` in [`GET /v1/models`](/docs/api-models)). A part the model does not take is a typed 400 that names the part. See [Content parts](#content-parts). |
| `stream` | boolean | Server-sent events. See [Streamed response](#streamed-response). |
| `max_tokens` | number | Output cap, from 1 to the model's own maximum (`max_output_tokens` in the catalog). The maximum is 128,000 on the Claude 5, GPT-6 and GPT-5.6 models, and can be below 32,768 (GPT-4o: 16,384). **If you send neither `max_tokens` nor `max_completion_tokens`, the gateway sets the model's own maximum** on the forwarded request, on every provider. It lowers the value when needed so that your input and the output fit the context window. If the catalog declares no maximum, or does not list the model (on your own connection), the value is 32,768. `null` counts as not set. A value under 1 (`0`, `-1`) or above the maximum is a 400. |
| `max_completion_tokens` | number | An alias of `max_tokens`. Both with different values is a 400. |
| `temperature` | number | Passed through. A value above 1 cannot be translated to an Anthropic-dialect provider. See [Wire compatibility](/docs/compat). |
| `top_p` | number | Passed through. |
| `stop` | string \| string[] | Becomes Anthropic `stop_sequences` on cross-dialect routes. |
| `tools` | array | `type: "function"` tools work on every model. A tool without `parameters` gets an empty object schema on Anthropic-dialect routes. Custom (free-form) tools work on the GPT-6 and GPT-5.6 models (all seven call OpenAI's Responses API): `{"type":"custom","custom":{"name":"…","description":"…","format":{"type":"text"}}}`, or a `grammar` format in `lark` or `regex` syntax. The call comes back as a `tool_calls` entry `{id, type: "custom", custom: {name, input}}`. Claude models take function tools only: a custom tool to a Claude model is a 400 that names it. See [Wire compatibility](/docs/compat#gpt-6-and-gpt-56-on-the-chat-surface). |
| `tool_choice` | string \| object | `"auto"`, `"none"`, `"required"`, `{"type":"function","function":{"name":"..."}}`, or, on the models above, `{"type":"custom","custom":{"name":"..."}}`. |
| `parallel_tool_calls` | boolean | `false` becomes `disable_parallel_tool_use` on Anthropic-dialect routes. |
| `n` | number | **A value above 1 is a 400 `invalid_request`.** See [Why `n > 1` is rejected](#why-n-1-is-rejected). |
| `stream_options` | object | Usage is always on. An explicit `{"include_usage": false}` is honored: you are billed the same, but you get no usage chunk. Otherwise an OpenAI-dialect provider gets the object with `include_usage` set. On a translated route the gateway acts on it itself, and does not forward it or name it in `x-tm-dropped-params`. |
| `provider` | object | Routing controls: `require_parameters`, `order`, `only`, `ignore`, `allow_fallbacks` and the rest. The gateway reads them and never forwards them. An unknown control is a 400. See [Routing policies](/docs/routing-policies). |
| `session_id` | string | A session id. The gateway sends it to OpenAI and Azure as `prompt_cache_key`, and to Morph as the header `x-session-id`. The headers `x-session-id`, `x-session-affinity` and `x-claude-code-session-id` work too. See [Sessions and prompt caching](/docs/compat#sessions-and-prompt-caching). |
| `prompt_cache_key` | string | OpenAI's cache key, sent unchanged to OpenAI and Azure. |
| `prompt_cache_retention` | string | `"in_memory"` or `"24h"`. Sent unchanged to OpenAI and Azure, dropped and recorded elsewhere. |
| `safety_identifier` | string | Your end-user id. On a house route the gateway sends a hash of it with your organization. See [Sessions and prompt caching](/docs/compat#sessions-and-prompt-caching). |
| `cache_control` | object | Automatic prompt caching, as on the Anthropic API. Sent to Anthropic, dropped and recorded on OpenAI and Azure. A field that Anthropic would refuse is dropped and recorded on every route (see [Wire compatibility](/docs/compat)). A `cache_control` mark on a `text` part is kept for Anthropic and passes unchanged to OpenAI and Azure, which ignore it. |

A top-level `api_key`, `base_url`, `connection_id` or other credential or destination field is a 400. You set keys and endpoints in the console, never in a request.

### Everything else

The table above is the **portable** set: it works on any provider. Other parameters depend on the dialect of the provider that serves the request:

| Serving provider | What happens |
|---|---|
| OpenAI dialect | The OpenAI chat parameters (`response_format`, `reasoning_effort`, `seed`, `user`, `logprobs`, `top_logprobs`, `frequency_penalty`, `presence_penalty`, `logit_bias`, `metadata`, `store`, `prediction`) reach it as you sent them. |
| Translated to the Anthropic dialect | `response_format`, `prediction`, `audio` and `modalities` are a typed 400, because they change what the model produces. `reasoning_effort` becomes Claude's own effort (`output_config.effort`). A Claude model that takes no effort drops it and records the drop. Other unknown parameters are dropped, never changed into something else. |

- Any other top-level key is dropped and recorded on every route.
- The `x-tm-dropped-params` header names every drop.
- `"provider": {"require_parameters": true}` turns a drop into a 400.
- The full tables are on [Wire compatibility](/docs/compat).

### Content parts

The gateway never drops content, because the provider would still bill for it. A part the model does not take is a typed 400 that names the field (for example `messages[0].content[1]`), on every route, before anything is billed or sent. An image to a text-only model is one such case.

- On an OpenAI-dialect provider, a part that the model takes passes through as sent.
- On a route translated to the Anthropic dialect (Claude), the gateway translates an `image_url` part (a JPEG, PNG, GIF or WebP data URL, or an `http(s)` URL) and a `file` part with a PDF data URL.
- On those routes, audio parts, video parts, files given by id, other file types and unknown part types are a typed 400 that names the field.

### Why `n > 1` is rejected

The gateway returns one choice. It refuses `n` above 1 before dispatch, so you never pay for fewer choices than you asked for:

```json
{
  "error": {
    "code": "invalid_request",
    "message": "n>1 is not supported in v1; request a single choice",
    "type": "invalid_request_error",
    "metadata": { "error_type": "invalid_request" }
  },
  "request_id": "..."
}
```

## Non-streamed response

A standard `chat.completion` object.

- On a translated response, `id` is `chatcmpl-<request-id>`.
- On an OpenAI-dialect provider, the body passes through with the provider's own `id`, keeping OpenAI's fields and the extensions below only. Fields a provider adds of its own (its cost, its token accounting, its host's name) are dropped. The message passes as the model wrote it ([what an answer keeps](/docs/compat#what-an-answer-from-an-openai-dialect-provider-keeps)).
- `model` names the model you asked for, also when the provider knows it under another id.
- For the `/v1/generation?id=` audit, use the `x-request-id` response header. It carries the bare request id on every route.
- On a [closed dedicated endpoint](/docs/dedicated-endpoints#closed-endpoints), the body keeps a fixed set of fields on every route. `model` is the endpoint id, `id` is `chatcmpl-` and the request id without dashes, and provider fields such as `system_fingerprint` and `native_finish_reason` are dropped.

```json
{
  "id": "chatcmpl-1c9a7b2e-...",
  "object": "chat.completion",
  "created": 1757000000,
  "model": "claude-sonnet-4-5",
  "choices": [
    {
      "index": 0,
      "message": { "role": "assistant", "content": "Hello!" },
      "finish_reason": "stop",
      "native_finish_reason": "end_turn"
    }
  ],
  "usage": {
    "prompt_tokens": 12,
    "completion_tokens": 5,
    "total_tokens": 17,
    "prompt_tokens_details": { "cached_tokens": 0, "cache_write_tokens": 0 },
    "cost": 0.000111
  }
}
```

Usage fields (the billing contract, see [Wire compatibility](/docs/compat)):

| Field | Meaning |
|---|---|
| `prompt_tokens` | All billed input tokens, **cache included**: uncached input + cache reads + cache writes. |
| `prompt_tokens_details.cached_tokens` | Cache reads, billed at the cached rate. |
| `prompt_tokens_details.cache_write_tokens` | Cache writes, billed at their own rate. Always present on billed responses, 0 when none. |
| `completion_tokens` | Output tokens. Reasoning tokens are a subset, in `completion_tokens_details.reasoning_tokens` when the provider reports them (streams always carry the field). |
| `cost` | Settled cost in USD, on billed keys only. The gateway computes it with the same integer micro-USD math as the ledger. It covers the **full request** debit, including attempts that failed over before your answer started, so you can recompute your bill from the response. |

`finish_reason` is one of `stop`, `length`, `tool_calls`, `content_filter`. On a response translated from an Anthropic-dialect provider, `choices[0].native_finish_reason` keeps the provider's raw stop reason (for example `end_turn`, `stop_sequence`). When the provider returns the model's thinking, it is in `message.reasoning_content`.

## Streamed response

With `stream: true`, the gateway sends SSE frames in `chat.completion.chunk` shape, in this order:

1. A role delta (`{"role":"assistant","content":""}`) as soon as the provider answers.
2. Content deltas: `delta.content`, `delta.reasoning_content` (thinking models), and `delta.tool_calls` fragments addressed by `index`. The id and `function.name` arrive on each call's first fragment.
3. A finish chunk with `finish_reason` set, plus `native_finish_reason` on translated streams.
4. A usage chunk with empty `choices`. It is always sent, without `stream_options`:

```text
data: {"id":"chatcmpl-...","object":"chat.completion.chunk","created":1757000000,"model":"claude-haiku-4-5","choices":[],"usage":{"prompt_tokens":12,"completion_tokens":5,"total_tokens":17,"prompt_tokens_details":{"cached_tokens":0,"cache_write_tokens":0},"completion_tokens_details":{"reasoning_tokens":0},"cost":0.000037}}

data: [DONE]
```

5. `data: [DONE]`.

- These are the shapes of a translated stream.
- On an OpenAI-dialect provider, the gateway relays the stream record for record and adds `usage.cost` to its usage chunk. Each chunk keeps OpenAI's fields only and names the model you asked for.
- On a closed dedicated endpoint, each chunk keeps only the fixed set of fields, and the provider's own SSE comment lines are dropped.
- When a started stream is silent for 15 s or more (slow reasoning models), the gateway sends an SSE comment (`: processing`) so that proxies keep the stream open. SDKs ignore it.
- See [Streaming](/docs/streaming) for the full detail.

> [!WARNING]
> If the serving provider fails **after** your answer started, the stream ends with one error chunk. It is a full chunk envelope with `finish_reason: "error"` and a top-level `error` object with `metadata.error_type`. **No `[DONE]` follows an error.** The gateway never restarts an answer on another provider mid-stream. Failover happens only before the first byte of output, and you do not see it. The same chunk ends a stream that reaches the request deadline (450 s from dispatch, or a routing policy's `timeout_ms`).

## Errors

Every error uses the OpenAI envelope with a stable `error.metadata.error_type` (`auth`, `rate_limit`, `insufficient_quota`, `model_unavailable`, `invalid_request`, `context_overflow`, `content_policy`, `upstream_error`, `gateway_error`, ...). [Errors](/docs/errors) has the full table with fixes.

- The gateway never relays a provider's raw error body. When a provider rejected the request, the class says so. The message has the provider's own words (up to 1,000 characters, without keys, account ids and host names), and the `x-tm-upstream-status` header has its status.
- A prompt too long for the model is OpenAI's standard 400. `error.code` is `context_length_exceeded`, with OpenAI's wording ("This model's maximum context length is M tokens. However, your messages resulted in N tokens. …"), so coding agents compact and retry.
- Out of credits is the OpenAI-standard **429 `insufficient_quota`**, which your SDK recognizes. It covers an empty balance or a monthly spend cap. On a cap breach, the message and the `x-tm-cap-reset` header give the UTC reset time.
- First trial keys, shown after verified sign-in, have a 20 RPM limit. Until an organization buys credit, all its keys together run at 20 RPM. The 429 `rate_limit` carries `retry-after`.

> [!TIP]
> A billed request reserves an estimated cost before dispatch: one token per byte of input, plus 65,536 per image part, and all of `max_tokens` (at most 32,768 tokens when you omit it). If your balance cannot cover it, with your organization's other in-flight reservations, you get `insufficient_quota` before any provider is called. You pay the observed usage only, so a large `max_tokens` never costs extra. The gateway holds it in full, though, so on a small balance set `max_tokens` to what you need, or leave it out. See [Reserve, then settle](/docs/pricing#reserve-then-settle).

## Response headers

| Header | Meaning |
|---|---|
| `x-request-id` | Gateway request id. Joins `/v1/usage` and `/v1/generation?id=`. |
| `x-tm-provider` | Deployment that served the request. `routerplus` on a closed dedicated endpoint. |
| `x-tm-attempts` | Dispatches made, failovers included. |
| `x-tm-upstream-status` | The provider's own HTTP status. |
| `x-tm-error-code` | Canonical error class, on failures. `x-tm-error-origin` says where the failure came from, and `x-tm-limit-scope` / `x-tm-limit-kind` name a limit that refused the request. See [Errors](/docs/errors). |
| `x-tm-upstream-model` | The id the gateway sent, when the provider knows the model under its own id. The response's `model` field still names the model you asked for. Absent on a closed dedicated endpoint. |
| `x-tm-served-by` | The provider that ran the request, when the route reports one. Absent on a closed dedicated endpoint. |
| `x-tm-dedicated-endpoint`, `x-tm-route-role` | On a [dedicated endpoint](/docs/dedicated-endpoints): its id, and `primary` or `fallback`. |
| `x-tm-queue-ms` | On a dedicated endpoint with a [paced limit](/docs/dedicated-endpoints#the-paced-limit): how long the request waited for its turn, in milliseconds. |
| `x-tm-auto-cache` | On a Claude route: `applied` when the gateway turned on Anthropic's automatic caching, else `skipped:` and the reason. Send the request header `x-tm-auto-cache: off` to turn it off. Absent on a closed dedicated endpoint. See [Automatic caching on Claude](/docs/compat#automatic-caching-on-claude). |
| `x-tm-dropped-params` | Comma-joined field paths of every parameter the gateway removed for this route. Absent when nothing was dropped. Send `"provider": {"require_parameters": true}` to get a typed 400 instead. |
| `x-tm-route-plan-id` | On billed requests: the id of the route plan that chose the deployments. |
| `x-tm-admission-mode`, `x-tm-remaining-rpm`, `x-tm-remaining-tpm` | On billed requests: how the request was admitted, and the smallest allowance left across the limits it claimed. See [Limits and capacity](/docs/admission). |

## Model discovery

`GET https://demo-api.routerplus.com/v1/models` (authenticated) returns the catalog in the OpenAI list shape, one row per model id, plus your organization's dedicated endpoints. The public directory with prices and context lengths is `https://demo.routerplus.com/api/models.json`.

- A model id that is not listed returns 404 `model_unavailable` with the id you asked for.
- If every deployment of a listed model is cooling down after failures, you get a 503 `gateway_error` with `retry-after: 5` and `x-tm-limit-kind: health`. Retry the same request.

See also: [POST /v1/messages](/docs/api-messages) · [Errors](/docs/errors) · [Wire compatibility & billing contract](/docs/compat) · [Quickstart](/docs/quickstart)