# Provider routing

Many models have more than one provider. RouterPlus picks the provider for each request. When a
provider cannot serve, RouterPlus fails over to the next one before the first byte reaches you.
Use the `provider` object to change the order or filter the providers.

## Request

```bash
curl https://demo-api.routerplus.com/v1/chat/completions \
  -H "Authorization: Bearer $TM_API_KEY" -H "content-type: application/json" \
  -d '{
    "model": "moonshotai/kimi-k3",
    "messages": [{ "role": "user", "content": "Hello" }],
    "provider": { "sort": "price", "max_price": { "prompt": 1, "completion": 4 } }
  }'
```

## Default order: lowest cost per good answer

For each provider of the model, RouterPlus estimates the cost of one good answer:

- the price of the model's typical request: uncached input, cached input at that provider's
  cache hit rate, and output;
- plus the cost of waiting: time to first token and output time at the provider's measured speed;
- plus the time lost to rate limits;
- divided by the share of requests that succeed, with a heavy penalty for provider errors and a
  penalty for malformed tool calls.

RouterPlus splits traffic as follows:

- Providers whose cost is close to the best share the traffic.
- Every provider keeps a small share, so RouterPlus keeps measuring new and recovering providers.
- A provider that fails or rate-limits gets less traffic until it recovers.
- Traffic above a provider's learned capacity goes to the next provider.
- Until providers have measurements, the cheapest goes first.

A conversation stays on the provider that served its first turn while that provider can serve.
This keeps the provider's prompt cache warm. Requests that share a long prompt prefix (system
prompt and tools) go to the same provider across sessions.

The response header `x-tm-routing` names the strategy: `routing_cost` (the default), `price`,
`throughput`, `latency` or `quality`. It adds `;pinned` or `;prefix` when a conversation pin or a
shared prefix chose the provider.

## The `provider` object

| Field | Effect |
|---|---|
| `order` | Try these providers first, in this order. Turns off the default order. |
| `only` | Only these providers. |
| `ignore` | Never these providers. |
| `allow_fallbacks` | `false`: only the first provider, with no failover. |
| `require_parameters` | Refuse the request instead of dropping a parameter a provider does not take. |
| `sort` | `"price"`, `"throughput"`, `"latency"` or `"quality"`, or `{ "by": ..., "partition": "model" \| "none" }`. Turns off the default order. See [Sort](#sort). |
| `preferred_min_throughput` | Tokens per second (a number means p50, or `{ "p50", "p75", "p90", "p99" }`). Slower providers go last. None is excluded. |
| `preferred_max_latency` | Seconds to first token, same forms. Slower providers go last. |
| `max_price` | `{ "prompt", "completion" }` in USD per million tokens, plus `request` and `image`. A provider above a cap is left out. |
| `quantizations` | Only providers that serve the model at these quantizations (`int4`, `int8`, `fp4`, `mxfp4`, `nvfp4`, `fp6`, `fp8`, `mxfp8`, `fp16`, `bf16`, `fp32`, `unknown`). |
| `zdr` | Only providers that keep no prompts or completions. |
| `data_collection` | `"deny"`: only providers that do not train on or keep request data. |
| `enforce_distillable_text` | Only `false` is accepted for now. |

[Guardrails](guardrails.md) on your key apply first. A request can narrow them. It cannot widen
them.

### Sort

| `sort` | Order |
|---|---|
| `price` | Cheapest first. |
| `throughput` | Highest measured tokens per second first. |
| `latency` | Lowest measured time to first token first. |
| `quality` | Highest share of well-formed tool calls first. Ties go by the default cost estimate. |

In the object form, `partition` is `"model"` (the default) or `"none"`. Both give the same
order, because RouterPlus routes one model per request.

After the sort, a conversation pin or a shared prompt prefix can still choose the provider.
`preferred_min_throughput` and `preferred_max_latency` then move providers that miss them to the
end.

### Errors

An unknown field in `provider`, or a `sort` or `partition` value outside the lists above, returns
`400`.

## Your default

An organization or workspace can make price its default order with
`PATCH /v1/org {"default_sort": "price"}` or `PATCH /v1/workspaces/<id>` (see the
[management API](api-management.md)). A `sort` in the request wins over this default.

## Rankings and benchmarks

| Endpoint | Returns |
|---|---|
| `GET /v1/rankings?period=day\|week\|month` | Models ranked by tokens served on RouterPlus. A model used by fewer than 3 organizations is not listed. |
| `GET /v1/rankings?task=code\|reasoning\|extraction\|chat\|creative\|vision\|long_context` | Models ranked for one kind of request. RouterPlus labels each request with its kind (the `task` label). |
| `GET /v1/rankings?view=sessions` | Models ranked by sessions, with the median cost and the median number of requests per session. |
| `GET /v1/benchmarks/<model>` | What the model takes (context, output limit, inputs, reasoning efforts, response formats) and how it serves now across its providers: the best latency, throughput, error rate and tool-call quality measured. |
| `GET /v1/benchmarks?task=coding\|math\|reasoning&limit=20` | Catalog models ranked by a quality index. See below. |
| `GET /v1/models/<model>/endpoints` | Each provider with its price, data policy, status and measurements. Its `quality` field holds the model's quality scores. |

`GET /v1/benchmarks?task=...` ranks `coding` by the coding index, `math` by the math index and
`reasoning` by the intelligence index. `limit` is 1 to 100 (default 20). When a model has no
coding or math index, the mean of its published coding or math evaluations (out of 100) ranks
it. A model with neither is not listed.

### Quality scores

`GET /v1/benchmarks/<model>` and `GET /v1/models/<model>/endpoints` carry a `quality` object:

```json
{ "quality": {
    "intelligence_index": 58.4, "coding_index": 55.0, "math_index": 90.1,
    "evaluations": { "gpqa": 0.9, "hle": 0.4, "livecodebench": 0.8 },
    "aa_model": { "slug": "claude-opus-5-5", "name": "Claude Opus 5.5 (Max)" },
    "source": "Artificial Analysis",
    "source_url": "https://artificialanalysis.ai/",
    "model_url": "https://artificialanalysis.ai/models/claude-opus-5-5",
    "fetched_at": "2026-10-31T09:00:00.000Z" },
  "scores_status": "ready" }
```

- The scores come from [Artificial Analysis](https://artificialanalysis.ai/). RouterPlus does not
  run these evaluations.
- `evaluations` holds every other evaluation Artificial Analysis publishes for the model, by its
  name.
- `quality` is `null` when Artificial Analysis has no entry for the model.
- `scores_status` is `ready`, `loading` or `unavailable`. `loading` means RouterPlus is fetching
  the scores. Ask again shortly.
- Scores refresh every 6 hours. `fetched_at` gives the time of the copy in use.
- When you show these scores, credit Artificial Analysis and link to
  https://artificialanalysis.ai/. Artificial Analysis asks for this attribution.