# Wire compatibility

This page defines how the gateway forwards, translates and drops parameters between the OpenAI and Anthropic formats, and how it counts the tokens you are billed for. A change to these rules is a change to your billing contract.

## Two surfaces, one catalog

| Surface | Endpoint | Auth |
|---|---|---|
| OpenAI-compatible | `POST https://demo-api.routerplus.com/v1/chat/completions` | `Authorization: Bearer` or `x-api-key` |
| Anthropic-compatible | `POST https://demo-api.routerplus.com/v1/messages` | `Authorization: Bearer` or `x-api-key` |
| OpenAI Responses | `POST https://demo-api.routerplus.com/v1/responses` | same |
| OpenAI Images | `POST https://demo-api.routerplus.com/v1/images/generations` | same |

- Every chat model in the catalog answers on both chat surfaces.
- Image models answer on the Images surface only.
- The Responses surface passes OpenAI's models through. Other chat models are a preview there ([POST /v1/responses](/docs/api-responses)).
- `GET https://demo-api.routerplus.com/v1/models` lists what your key can serve, in the OpenAI shape, or the Anthropic shape with an `anthropic-version` header.

Each request takes one of two routes:

- **Passthrough route:** your format matches the provider's dialect. The gateway forwards your dialect's parameters as sent, and adds `stream_options.include_usage` to a streamed OpenAI-dialect dispatch. The provider decides what it accepts, including Anthropic prompt caching (`cache_control`), `thinking`, `top_k` and the `anthropic-beta` values below. A top-level key outside your dialect's parameter set is dropped and recorded. Session ids and cache marks follow [Sessions and prompt caching](#sessions-and-prompt-caching).
- **Translated route:** the dialects differ. The gateway translates the request, the stream and the errors, by the rules below.

A "typed 400" below is decided per deployment. If another deployment of the model can serve the request (it speaks your dialect, or takes a content part), the request goes there. You get the 400 only when no deployment can. See [Content is never silently dropped](#content-is-never-silently-dropped).

## Parameter handling on passthrough routes

Any other top-level key is dropped and recorded.

| Dialect | Forwarded as sent |
|---|---|
| OpenAI → OpenAI-dialect provider | `model`, `messages`, `stream`, `stream_options`, `temperature`, `top_p`, `max_tokens`, `max_completion_tokens`, `stop`, `n`, `tools`, `tool_choice`, `parallel_tool_calls`, `response_format`, `reasoning_effort`, `seed`, `user`, `logprobs`, `top_logprobs`, `frequency_penalty`, `presence_penalty`, `logit_bias`, `metadata`, `store`, `prediction` |
| Anthropic → Anthropic-dialect provider | `model`, `messages`, `system`, `max_tokens`, `stream`, `temperature`, `top_p`, `top_k`, `stop_sequences`, `tools`, `tool_choice`, `metadata`, `thinking`, `output_config`, `cache_control`, `context_management` (to Anthropic's own API; on Bedrock it is dropped and recorded) |

The GPT-6 and GPT-5.6 models are translated to OpenAI's Responses API (see [GPT-6 and GPT-5.6 on the chat surface](#gpt-6-and-gpt-56-on-the-chat-surface)).

### What an answer from an OpenAI-dialect provider keeps

A chat completion or stream chunk from an OpenAI-dialect provider keeps only OpenAI's fields and the gateway's documented extensions. Other fields a provider adds at the top level, on a choice or in `usage` (its own cost, token accounting or host name) are dropped.

| Where | Kept |
|---|---|
| Top level | `id`, `object`, `created`, `model`, `choices`, `usage`, `system_fingerprint`, `service_tier`, `obfuscation`, `error` |
| Each choice | `index`, `message` or `delta`, `finish_reason`, `logprobs`, `native_finish_reason`, `error` |
| `usage` | `prompt_tokens`, `completion_tokens`, `total_tokens`, `prompt_tokens_details` (`cached_tokens`, `audio_tokens`, `cache_write_tokens`), `completion_tokens_details` (`reasoning_tokens`, `audio_tokens`, `accepted_prediction_tokens`, `rejected_prediction_tokens`), and `cost`: the gateway's, never the provider's |

- The message and delta pass as written: content, reasoning (`reasoning_content`, `reasoning`, `reasoning_details`), tool calls and annotations.
- `model` names the model you asked for. The provider's own id, if different, is in the `x-tm-upstream-model` header. A provider sent your own id can answer with a dated snapshot (`gpt-4o-mini-2024-07-18`).
- Through your own provider key ([BYOK](/docs/byok)), the answer passes as sent.
- A [closed dedicated endpoint](/docs/dedicated-endpoints#closed-endpoints) keeps a smaller set of fields.

### Provider-specific keys

- The `reasoning` object (`{"enabled": false}`, `{"effort": "low"}`) is dropped and recorded. Send `reasoning_effort`. DeepInfra takes `"none"` (its `"enabled": false`), `"low"`, `"medium"` and `"high"`.
- `prompt_cache_retention` and `safety_identifier` go to OpenAI and Azure only.
- A `cache_control` mark on a content part passes unchanged to every OpenAI-dialect provider. OpenAI and Azure ignore it.
- On Bedrock, a top-level `cache_control` becomes a mark on the last block that can carry one.
- On every route, a top-level `cache_control` that Anthropic would refuse is dropped and recorded as `cache_control`. Anthropic refuses:
  - a field other than `{"type": "ephemeral"}` with an optional `ttl` of `"5m"` or `"1h"`
  - a fifth mark
  - a 1-hour field after a 5-minute mark
  - a field whose TTL differs from the mark on the last block
- `mcp_servers` and `container` are a typed 400 on every route.

### The `anthropic-beta` header

The gateway forwards to Anthropic-dialect providers only the `anthropic-beta` values that do not change how a token is billed:

- On every Anthropic-dialect provider: the prefixes `prompt-caching`, `token-efficient-tools`, `fine-grained-tool-streaming`, `interleaved-thinking`, `claude-code`, `oauth` and `computer-use`.
- On Anthropic's own API, also these values for Claude Code on the Claude 5 models:
  - the prefixes `per-turn-control` and `mid-conversation-output-config` (effort inside a message)
  - `context-management` (context editing; compaction's own beta is not forwarded)
  - `mid-conversation-tool-changes` and `inline-tools` (tools added during a conversation)
  - `thinking-display-updates` (readable progress updates)

Any other value is dropped and recorded as `header:anthropic-beta:<value>`. Examples: `advisor-tool` (a second model at its own rates), and the auto-mode values `dangerous-tool-use` and `afk-mode` with the `safeguards` field (their result field is not passed back).

On Anthropic's direct API, Claude Code's `thinking: {"type":"adaptive","display":"updates"}` goes with its `thinking-display-updates-2026-08-18` beta, and progress updates and signatures survive streaming. Bedrock drops `thinking.display: "updates"` and that beta, and keeps the thinking mode. Both drops are reported, and `require_parameters` refuses the request.

## Parameter handling on translated routes

### OpenAI surface → Anthropic-dialect provider

| Parameter | Handling |
|---|---|
| `model`, `messages`, `stream`, `temperature`, `top_p` | Translated or copied as sent |
| `max_tokens`, `max_completion_tokens` | Translated. With neither, the gateway sets the model's declared maximum output (32,768 when none is declared), fitted to its context window. See [POST /v1/chat/completions](/docs/api-chat-completions) |
| `stop` (string or array) | → `stop_sequences` |
| `system` / `developer` messages | → the `system` string, joined with blank lines. If a part has a `cache_control` mark, `system` becomes text blocks, and each mark ends a block |
| `cache_control` on a text part | Kept on the Anthropic block. A `role:"tool"` message's mark goes on its `tool_result` |
| Other keys of a text part | Not forwarded, not recorded |
| `cache_control` (top-level) | → the Anthropic top-level field (automatic caching). On Bedrock it becomes a mark on the last block |
| `tools`, `tool_choice` | `auto`→`{type:"auto"}`, `required`→`{type:"any"}`, `{function:{name}}`→`{type:"tool",name}`, `none`→`{type:"none"}`. With `none` and no tools in the history, tools are left out, so you do not pay for them. A tool without `parameters` gets an empty object schema. The Claude 5 models refuse `any` and `tool` (a forced tool choice) with a 400 |
| `role:"tool"` messages | → `tool_result` blocks. Consecutive results merge into one Anthropic user message |
| Tool-call ids | Kept when valid for Anthropic (letters, digits, `_`, `-`, at most 64 characters). Others become `toolu_` and a hash of the id, the same in the call, its result and every turn, so pairs match and the cached prompt start is stable. Answers carry the provider's own ids |
| `image_url` parts in user messages | → `image` blocks. A base64 data URL (JPEG, PNG, GIF or WebP, typed by its bytes) becomes a base64 source. An `http(s)` URL becomes a URL source that Anthropic fetches. A host that refuses Anthropic's fetcher gives a 400, with no failover. A `cache_control` mark is kept. A `detail` other than `auto` is dropped and recorded. Another image type is a typed 400 naming the part |
| `file` parts in user messages | A PDF sent as a data URL → a `document` block, with the file's `filename` as its title. A `file_id`, or a file of another type, is a typed 400 naming the part |
| An assistant message with `content: null` and no `tool_calls` (OpenAI's refusal shape) | Dropped and recorded |
| `stream_options` | Usage is on. An explicit `include_usage: false` removes the usage chunk, and you are still billed. It is not forwarded and not named in `x-tm-dropped-params` |
| `n` | `n > 1` is a 400 `invalid_request` on `/v1/chat/completions` and `/v1/messages`. The images route takes `n` up to 4 (see [/docs/api-images](/docs/api-images)) |
| `temperature` | Forwarded when ≤ 1. `temperature > 1` is a typed 400, never clamped (Anthropic's range is 0 to 1) |
| `top_p` | Forwarded only without `temperature` (Anthropic takes one of the two). Otherwise dropped and recorded |
| `parallel_tool_calls: false` | → `tool_choice.disable_parallel_tool_use: true` |
| `tools[].function.strict` | Mapped 1:1 |
| `response_format`, `prediction`, `audio`, `modalities` | **Typed 400** naming the parameter. Not mapped across dialects yet |
| `reasoning_effort` | → `output_config.effort` at the same level (`low` to `max`; `minimal` → `low`). `none` → `thinking: {"type": "disabled"}` on Claude Haiku 5.5, Sonnet 5 and Opus 5, and `low` on the other Claude 5 models. Without it, thinking is the model's default. On Claude Sonnet 4.5 and Haiku 4.5 it is **dropped and recorded** (`none` is not recorded). A value outside OpenAI's levels is a **typed 400** |
| Trailing `assistant` message | **Typed 400** (an assistant prefill, which current Claude models reject) |
| Everything else (`logprobs`, `seed`, penalties, …) | **Dropped and recorded**, never forwarded |

### Anthropic surface → OpenAI-dialect provider

| Parameter | Handling |
|---|---|
| `model`, `stream`, `temperature`, `top_p` | Copied as sent |
| `max_tokens` | → `max_completion_tokens` (o-series and GPT-5 reasoning models reject `max_tokens`). When absent, set to the model's maximum, as above |
| `system` (string or text blocks) | → one leading `system` message |
| `stop_sequences` | → `stop` |
| `messages` | `tool_use` ↔ `tool_calls`, `tool_result` blocks → `role:"tool"` messages (one per result). A `system` message inside the conversation stays at its place. See [below](#reasoning-and-structured-output-on-openai-dialect-routes) |
| `image` blocks in user messages | → `image_url` parts, from a base64 source or a URL source. An image in an assistant message or a tool result, or an image given by a file id, is a typed 400 naming the block |
| `document` blocks in user messages | A base64 PDF → a `file` part. The block's `title`, `context` and `citations` are dropped and recorded. A document given by URL or as text is a typed 400 naming the block |
| `tools`, `tool_choice` | `auto`→`"auto"`, `any`→`"required"`, `none`→`"none"`, `{type:"tool",name}`→`{function:{name}}`. `disable_parallel_tool_use: true` → `parallel_tool_calls: false`. `tools[].strict` maps 1:1. An empty `tools` list is sent as no list, without `auto`, `none` or the parallel flag. Server tools (computer use, web search, bash, text editor) are a **typed 400** |
| `thinking` | `adaptive`, `enabled` → the model's own reasoning stays on (`budget_tokens` **dropped and recorded**). `disabled` → `reasoning_effort: "none"`. A `type` or `display` value the gateway does not know is a **typed 400**. See [below](#reasoning-and-structured-output-on-openai-dialect-routes) |
| `output_config` | `effort` → `reasoning_effort`. `format` `json_schema` → a strict json_schema `response_format`. Another `format` type is a **typed 400** |
| `top_k`, `metadata` | **Dropped and recorded**. On a house route, `metadata.user_id` becomes the end-user id and is not recorded (see [Sessions and prompt caching](#sessions-and-prompt-caching)) |
| `cache_control` (nested and top-level) | **Dropped and recorded with the full path**, for example `messages[0].content[0].cache_control` |
| `tool_result.is_error` (nested) | **Dropped and recorded with its full path** |

> [!NOTE]
> Every drop is **recorded** per request, with its full path, in the `x-tm-dropped-params`
> header, in each attempt's `dropped_params` in [`GET /v1/generation`](/docs/api-usage),
> and in the ledger (which holds no content). With `"provider": {"require_parameters":
> true}`, the first drop is a typed 400 instead. Only parameters are dropped, never content.

### Reasoning and structured output on OpenAI-dialect routes

When an OpenAI-dialect provider serves an Anthropic-format request (GLM 5.3, Kimi K3, DeepSeek and the other open models), reasoning and output controls map to these OpenAI fields:

| You send | The provider gets |
|---|---|
| `thinking: {"type": "adaptive"}` or `{"type": "enabled", ...}` | Nothing: the model reasons by default. `budget_tokens` is dropped and recorded |
| `thinking: {"type": "disabled"}` | `reasoning_effort: "none"`. An `output_config.effort` beside it is dropped and recorded |
| `output_config.effort` | `reasoning_effort` at the same level, or the nearest level a provider declares. A system message's own effort applies from its place on, so the last one wins |
| `output_config.format` with `type: "json_schema"` | `response_format: {"type": "json_schema", "json_schema": {"name": "output", "schema": <your schema>, "strict": true}}`, only on providers that take json_schema. If none does, a typed 400 names `output_config.format` before anything is held or sent |
| A `system` message inside `messages` | A `system` message at the same place. A turn-scoped one (`clear_at: "next_user_message"`) is left out once a user message follows it. One with no content only sets effort. A tool added or removed in one that still shows is a typed 400 |

`thinking.display` sets how reasoning comes back. An open model's reasoning is its raw chain of thought.

- `"omitted"` and `"updates"`: each `thinking` block has an empty `thinking` field, as on Anthropic's API, and the text travels in its `signature`. Send the block back unchanged: Morph's and Mistral's routes read the reasoning from it. Under `"updates"`, Claude Code shows such a block as a progress line, without the raw reasoning.
- `"summarized"` or no `display`: readable `thinking` blocks with the reasoning text.

Dropped and recorded:

- `thinking` and the effort, when a provider serves the model with thinking off (for example Qwen3.8 Max on DeepInfra) or the model does not reason (Mistral's Codestral, Large 3 and Ministral 3).
- `context_management` (context editing) and `safeguards` (Claude Code's auto-mode classifier context).

### GPT-6 and GPT-5.6 on the chat surface

The GPT-6 and GPT-5.6 models (GPT-6 Astra, GPT-6.1 Sol, GPT-6 Sol, GPT-6 Luna, GPT-5.6 Sol, GPT-5.6 Terra and GPT-5.6 Luna) take tools only on OpenAI's Responses API. You call them on `/v1/chat/completions` or `/v1/messages`, and the gateway translates. This applies on the marketplace route and on your own OpenAI connection.

| Chat field | Handling |
|---|---|
| `messages` | → Responses input items: text, image and file parts; an assistant's tool calls as `function_call` or `custom_tool_call` items; a tool message as the matching output item. Audio parts are a typed 400 |
| `max_tokens`, `max_completion_tokens` | → `max_output_tokens`. A limit under 16 is sent as 16 (OpenAI's floor) and recorded as dropped |
| `tools` with `type: "function"` | → function tools. A tool without `strict` is sent with `strict: false`, the Chat Completions default |
| `tools` with `type: "custom"` | → custom (free-form) tools, with a `format` of `text` or a `grammar` in `lark` or `regex` syntax. Both the chat form (`{type: "custom", custom: {name, description, format}}`) and Cursor's flat form work. A call comes back as a `tool_calls` entry `{id, type: "custom", custom: {name, input}}`. Send its result as a `role:"tool"` message. `input` must be a string |
| `tool_choice` | → the Responses form. It may name a custom tool |
| Tool-call ids | An id over 64 characters becomes `call_` and a hash of it, the same in the call and its result |
| `response_format`, `reasoning_effort` | → their Responses forms |
| `store` | Sent as `false` unless you set it |
| `n`, `stop`, `seed`, `logprobs`, `top_logprobs`, `logit_bias`, `prediction`, the penalties | Dropped and recorded. A value equal to the default (`n: 1`, a penalty of 0) is dropped without a record |
| `temperature`, `top_p` | Dropped and recorded, except `temperature: 1` and `top_p: 1`. These models take only their default sampling |

The answer is a chat completion, or `chat.completion.chunk` records on a stream. `finish_reason` is `tool_calls` after a tool call and `length` at `max_output_tokens`. A short GPT-6 commentary before the answer is joined to it with a blank line. Reasoning does not carry from one turn to the next.

### Content is never silently dropped

A provider that drops a content part still bills the request, so the gateway never drops content.

**A part goes only to a model that takes it.**

- Each listing declares what a message may carry: text, plus any of image, file (a document such as a PDF), audio and video. `GET /v1/models` shows it for your key in `architecture.input_modalities`. A model's page lists only the inputs that every listing takes.
- The gateway sends a part only to a deployment whose listing takes its kind. A part that no deployment takes is a **typed 400 naming the exact field**, on both surfaces, before anything is reserved or sent: `messages[0].content[1]: model "gpt-4.1-mini" does not take audio input; it takes text, image and file`.
- Checked parts: `image_url`, `file`, `input_audio` and `video_url` (or `input_video`) on the OpenAI surface; `image` and `document` blocks on the Anthropic surface, also inside a `tool_result`.
- A BYOK route declares nothing and gets every part as sent.

**A translation names what it cannot carry.** Images (JPEG, PNG, GIF and WebP) and PDFs in user messages cross between the formats, as the tables show. On a translated route, this content is a **typed 400 naming the exact field** (`messages[2].content[0]` and so on):

- audio parts
- an image or a file given by file id
- a file other than a PDF
- an image in an assistant message or a tool result
- unknown block types

**Exception: echoed thinking blocks.** On the Anthropic surface, assistant `thinking` and `redacted_thinking` blocks that you echo back are dropped, not rejected. The gateway itself emits them for an OpenAI-dialect model's reasoning (on a stream, and as a leading block on a non-streamed reply).

- Thinking is never forwarded to an OpenAI-dialect provider.
- A signed `thinking` block (Claude's own) and a `redacted_thinking` block are named in `x-tm-dropped-params`.
- An unsigned block, or one with the gateway's own signature (text hidden by `thinking.display`), is dropped without a record, and `require_parameters` does not refuse the next turn for it.
- On an Anthropic-dialect provider, the gateway's own blocks are left out without a record, because Anthropic verifies every signature. A conversation served elsewhere while Anthropic was unavailable can continue on Anthropic.

## Sessions and prompt caching

A provider keeps a prompt's cache on one machine. The gateway passes on what each provider needs to route a conversation back to it.

**Session id.** The gateway takes the first valid value of these inputs, in this order:

| Order | Input | Where |
|---|---|---|
| 1 | `session_id` | Body |
| 2 | `x-session-id` | Header |
| 3 | `x-session-affinity` | Header |
| 4 | `x-claude-code-session-id` | Header |

- A valid value has 1 to 256 printable ASCII characters. The body field `prompt_cache_key` follows the same rule.
- An invalid value is ignored, with no error. The gateway uses the next input and records the ignored one as `session_id (invalid)`, `prompt_cache_key (invalid)` or `header:<name> (invalid)`. With `provider.require_parameters: true`, it is a 400.
- A valid session input is never recorded as dropped.

Each route gets its own field:

| Route | Field sent | Value |
|---|---|---|
| OpenAI (house or BYOK), Azure | `prompt_cache_key` | Your `prompt_cache_key`, else the session id. Nothing if you sent neither |
| Morph | Header `x-session-id` | The session id, else your `prompt_cache_key`. Nothing if you sent neither. Morph picks region and worker by it |
| Anthropic, Bedrock | Nothing | Its cache matches the prompt prefix |
| DeepInfra | Nothing | Its cache matches the prompt prefix without a key |

- Values go unchanged. A value longer than the provider accepts is replaced by `tm-` followed by 43 base64url characters: a hash of the value.
- `prompt_cache_retention` (`"in_memory"` or `"24h"`) goes unchanged to OpenAI and Azure. Other routes drop and record it.

**End-user ids on house routes.** A house route runs on the marketplace's provider account, shared by all customers. It gets one end-user id: a hash of your organization and your `safety_identifier` (else `user`, else, on `/v1/messages`, `metadata.user_id`), or of your organization alone.

| House route | Field sent | Not sent |
|---|---|---|
| `openai` | `safety_identifier` | `user` |
| `anthropic` | `metadata.user_id` | `user`, `safety_identifier` |

BYOK routes get your values unchanged: `user` and `metadata`, and `safety_identifier` on OpenAI and Azure from `/v1/chat/completions`. Anthropic and Bedrock BYOK routes, and any translation from `/v1/messages`, drop and record `safety_identifier`.

**Cache marks.**

- The gateway never adds a `cache_control` mark to a block or a part.
- On a request with no `cache_control` at all, the gateway may add the top-level field ([Automatic caching on Claude](#automatic-caching-on-claude)).
- The ledger prices every cache write at the 5-minute rate. On a house route, `ttl: "1h"` is removed from every mark and recorded as `<path>.cache_control.ttl`, for example `system[0].cache_control.ttl`. `ttl: "5m"` passes. BYOK routes keep `ttl: "1h"`.

**Staying on the first route.** When the gateway's quota pool for your first route is full, a follow-up request (an assistant or tool message after a user message) waits up to 5 seconds for that route, so it does not move to a cold cache. `x-tm-affinity-wait-ms` gives the milliseconds waited.

### Automatic caching on Claude

Coding agents that call Claude through an OpenAI-compatible setting (Cline, Cursor, Kilo Code, OpenCode and others) send no cache marks. The gateway turns on Anthropic's automatic caching when all of these are true:

1. Anthropic's own API serves it: the marketplace's `anthropic` route, or your own Anthropic connection. Not Bedrock or dedicated endpoints.
2. The request has no `cache_control` anywhere (top level, system prompt, tool, message, or a block inside a tool result).
3. It has tools, or an assistant turn before its last message. A single question without tools is left alone.
4. The prompt (system prompt, tools and messages, at four bytes a token) reaches the model's minimum cacheable length:
   - 512 tokens on Claude Opus 5.5, Opus 5, Fable 5.1, Fable 5, Sonnet 5.5 and Haiku 5.5
   - 1,024 on Claude Sonnet 5 and Sonnet 4.5
   - 4,096 on Claude Haiku 4.5
5. You did not turn it off with the request header `x-tm-auto-cache: off` (any letter case).

The gateway then adds only the top-level `cache_control: {"type": "ephemeral"}`. Anthropic puts one cache point on the last cacheable block and moves it forward as the conversation grows. The cache lasts 5 minutes, and the answer does not change. The first turn pays a cache write (1.25 times the input price). Later turns read at the cache-read price, a tenth of the input price or less.

A response from a Claude route carries `x-tm-auto-cache`: `applied`, or `skipped:` and the reason (`opted-out`, `client-markers`, `single-turn`, `short-prompt`, `bedrock`, `dedicated` and a few more). A closed dedicated endpoint never sends it. `x-tm-dropped-params` does not change.

**Cache privacy.** Marketplace requests reach Anthropic from one RouterPlus account, and Anthropic keeps caches per account. A cache read needs a prompt start that is byte for byte the same as an earlier one, which can be another customer's. So someone who knows your exact prompt start could tell from their own cache reads that it was sent in the last few minutes. This applies also to requests with their own marks. To avoid it, send `x-tm-auto-cache: off` and no marks.

## Usage accounting (how tokens are counted)

| Concept | OpenAI surface | Anthropic surface |
|---|---|---|
| Billable input | `prompt_tokens`, cache-inclusive: input + cache reads + cache writes | `input_tokens`, which excludes cache reads and writes |
| Cache reads | `prompt_tokens_details.cached_tokens` | `cache_read_input_tokens` |
| Cache writes | `prompt_tokens_details.cache_write_tokens` | `cache_creation_input_tokens` |
| Output | `completion_tokens` | `output_tokens` |
| Reasoning | `completion_tokens_details.reasoning_tokens` (a subset of output) | not reported separately |

Translation converts by this table. `total_tokens = prompt_tokens + completion_tokens`.

- **Usage is always on**, streamed or not. Only an explicit `include_usage: false` removes the usage chunk (see `stream_options` above).
- **Cache writes** bill at their own rate. On billed OpenAI-surface responses, `prompt_tokens_details.cache_write_tokens` is always present (0 when the provider reports none).
- **Every billed response carries `usage.cost` (USD)**, in the final usage chunk on a stream, else in the body. It uses the ledger's integer micro-USD math and covers the full debit, including attempts that failed over before your answer started.
- **Rounding.** Prices are micro-USD per million tokens. The reservation rounds up, and settlement rounds down.
- **Web search has a fee per search**, at the provider's price, on top of the tokens: Anthropic's web search server tool, and OpenAI's hosted web search on `POST /v1/responses`. The fee is inside `usage.cost`. See [Pricing & billing](/docs/pricing#web-search).

> [!WARNING]
> An Anthropic-surface stream that dies mid-answer never sends `message_delta`, its only
> usage slot. Get those requests from `GET https://demo-api.routerplus.com/v1/generation?id=REQUEST_ID`,
> which returns every attempt with tokens and settled cost. The OpenAI surface emits known
> usage and cost *before* the terminal error event.

## Streams

Event order on the OpenAI surface:

1. A role-priming delta, sent as soon as the upstream responds.
2. Content deltas.
3. The finish chunk.
4. The usage chunk.
5. `data: [DONE]`.

The gateway never sends `[DONE]` after an error. Nothing follows a terminal event.

- **Keep-alives.** After output starts, each silence of 15 seconds or more gets a keep-alive frame, so proxies do not close a slow stream: an SSE comment (`: processing`) on the OpenAI surface, a native `ping` event on the Anthropic surface. SDKs skip both.
- **The commit boundary.** Before output starts, the gateway fails over provider failures with zero backoff, and you see one response with one event sequence. `x-tm-attempts` counts the dispatches. After output starts, the gateway **never** switches providers. A failure becomes one terminal error event inside the 200 stream, and you decide whether to retry. A refusal is never rerouted.
- **The request deadline.** 450 seconds from dispatch, or a routing policy's `timeout_ms`. A stream still open then gets the same terminal error event.
- **Mid-stream terminal shape.** On the OpenAI surface: a full chunk envelope with `finish_reason: "error"` and a top-level `error` object. On the Anthropic surface: an `event: error` frame.

## `finish_reason` and `native_finish_reason`

Translated responses map finish reasons both ways:

| Anthropic `stop_reason` | OpenAI `finish_reason` |
|---|---|
| `end_turn` | `stop` |
| `stop_sequence` | `stop` |
| `max_tokens` | `length` |
| `tool_use` | `tool_calls` |
| `refusal` | `content_filter` |

Two stop reasons both become `stop`. Translated responses on the OpenAI surface carry the raw value in `choices[0].native_finish_reason`, in the body and in the stream's finish chunk. Use it for analytics. Passthrough responses are relayed as sent.

## Errors

The gateway writes every error body in the dialect you called with.

- Each body carries a stable `error_type` with the canonical class: `error.metadata.error_type` on the OpenAI surface, `error.error_type` on the Anthropic surface.
- Each body also carries the dialect's native fields, so SDK error handling and retries work. On the Anthropic surface these are the native type strings (`billing_error`, `rate_limit_error`, `authentication_error`, `not_found_error`, `api_error`, `invalid_request_error`).
- The gateway never relays a provider's raw error body. The message carries the provider's own message, up to 1,000 characters, with keys, account ids and host names removed (never on a closed dedicated endpoint).
- `x-tm-upstream-status` carries the provider's HTTP status.
- The body's `metadata` names where the failure came from (`origin`, and the limit that refused it when one did). See [Errors](/docs/errors).

A prompt too long for the model is a 400 in the surface's standard shape, which coding agents compact on: code `context_length_exceeded` with OpenAI's wording, or `prompt is too long: N tokens > M maximum` on the Anthropic surface. The class is `context_overflow` (see [Context overflow](/docs/errors#context-overflow)).

Canonical classes and how routing treats them:

| Upstream signal | `error_type` | Failover |
|---|---|---|
| 429 (Retry-After honored, capped at 59 s; 10 s when the provider names none) | `rate_limit` | yes |
| 401/402/403 from a provider (a problem with the RouterPlus account, not yours) | `auth` | yes |
| Context/length errors | `context_overflow` | no: fix the request or pick a bigger window |
| Content policy / refusal | `content_policy` | **never**: a refusal is not rerouted |
| 404 / model not found upstream | `model_unavailable` | yes |
| 5xx / 529 / overloaded | `upstream_error` | yes |
| Other 4xx | `upstream_error` | no |

On `POST /v1/decisions`, some signals map differently. The gateway reads them by their status and the provider's own error code, never by the provider's name:

- These are a 503 `model_unavailable` with `metadata.retryable: false`: Levanto's 402, Fastino's 402 or 403, a System One provider's 401, 402 or 403 on the RouterPlus account, and Cloudflare's 429 with code 3036 (the free daily allocation is used up).
- Cloudflare's 413 with code 5021 is a 413 `context_overflow`.
- On Perplexity Decider v1 27B, Perplexity's 400 that names the model's maximum context length is a 400 `context_overflow`, by the context rule above.
- Timeouts: a System One provider's 408 is a retryable 504 `upstream_error`.
- OpenAI's 404 `model_not_found` on GPT-6 Luna Decisions means RouterPlus's access to the model was withdrawn. It is a 404 `model_unavailable` by the 404 rule above, with no second provider to try.
- A decisions provider's 429 never counts against the deployment's health circuit.

On GPT-6 Luna Decisions, OpenAI can decline one question and answer the rest. That is not a `content_policy`
error. The response is a 200, that question's answer is `{"type": "refusal"}`, and the call is billed. See [POST /v1/decisions](/docs/api-decisions).

Out of credits is a 429 `insufficient_quota`, which OpenAI's SDKs back off on. Error bodies never echo flagged input content. [Errors](/docs/errors) lists the fix for each status.

## Response headers

| Header | When | Meaning |
|---|---|---|
| `x-request-id` | every call | the id that `GET /v1/generation?id=` audits |
| `x-tm-provider` | served requests | the deployment that served the answer |
| `x-tm-attempts` | served or failed dispatches | physical dispatches, failovers included |
| `x-tm-upstream-status` | when an upstream responded | the provider's own HTTP status |
| `x-tm-error-code` | failures | canonical class |
| `x-tm-error-origin` | limit, provider and infrastructure failures | where it came from: `gateway_admission`, `upstream_quota`, `upstream`, `gateway_infrastructure` or `authorization` |
| `x-tm-limit-scope`, `x-tm-limit-kind`, `x-tm-limit-id` | a refused limit | which limit refused the request (see [Limits and capacity](/docs/admission)) |
| `x-tm-cap-reset` | spend-cap 429s | when the monthly cap resets (the first of next UTC month) |
| `x-should-retry` | spend-cap 429s | `false`: do not retry before `x-tm-cap-reset`. OpenAI's and Anthropic's SDKs and Claude Code obey it |
| `x-tm-auto-cache` | a Claude route served the request (never on a closed dedicated endpoint) | `applied`, or `skipped:` and the reason (see [Automatic caching on Claude](#automatic-caching-on-claude)) |
| `x-tm-dropped-params` | when not empty | comma-joined names of the parameters the gateway removed. Every drop is recorded |
| `x-tm-upstream-model` | when the route knows the model by another id | the id the gateway sent upstream |
| `x-tm-served-by` | when a route reports one | the provider that ran the request |
| `x-tm-route-plan-id` | billed requests | the id of the route plan that chose the deployments |
| `x-tm-admission-mode` | billed requests | `shared`, or `bounded_local` while the shared limit store is unreachable |
| `x-tm-remaining-rpm`, `x-tm-remaining-tpm` | admitted requests | the smallest remaining allowance across the limits the request claimed |
| `retry-after` | 429/503 | whole seconds to wait, at most 59 (`usage_limited` sends 60 or 3,600). A provider's value is honored, capped at 59. A provider's 429 without one gets 10 |

Browsers can read every header above except `x-tm-error-code` and `x-tm-route-plan-id`, which are not CORS-exposed. In browser code, use the body's `error_type`.

Optional: send `HTTP-Referer` and `X-Title` to identify your app in per-app analytics. They carry no content.

## Not supported

- Audio and video input, and files other than PDFs, across dialects (see [Content is never silently dropped](#content-is-never-silently-dropped)).
- Image edits, variations and streaming (see [POST /v1/images/generations](/docs/api-images)).
- `n > 1` on the chat surfaces.
- Model id variants. An id that contains `:` is not in the catalog and returns 404. The gateway never substitutes another model.
- On passthrough routes, the provider decides how to handle forwarded parameters. RouterPlus guarantees only what this page documents.

See also: [Migrate](/docs/migration) · [Quickstart](/docs/quickstart) · [Errors](/docs/errors)