Dedicated endpoints

A dedicated endpoint is provider capacity reserved for your organization on one model, with its own limits, prices and failover rules. RouterPlus sets one up with you under a contract. There is no self-serve setup.

You call the endpoint with the model id RouterPlus gives you, in your organization's namespace: <your-namespace>/<name>, for example acme/gemma-31b.

Calling an endpoint

Use the endpoint's model id in place of a catalog model id. Both surfaces (/v1/chat/completions and /v1/messages), streaming, and every request field the model accepts work as usual.

curl https://dev2-api.routerplus.com/v1/chat/completions \
  -H "Authorization: Bearer $TM_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"model": "acme/gemma-31b", "messages": [{"role": "user", "content": "Hello"}]}'

Only your organization's keys can call the endpoint, and it can be limited to named workspaces. Any other key gets the same 404 model_unavailable as for an unknown model.

On an open endpoint, the response's model is the model that ran, for example "model": "google/gemma-4-31b-it". On a closed endpoint it is the endpoint id (see Closed endpoints). Two headers say how the request was served:

• In /v1/generation, each attempt's route_context.dedicated records the endpoint id, the revision of its configuration in force, and the role.
• GET /v1/models lists your organization's active endpoints with owned_by routerplus, on the base URL that serves the endpoint.
• The Endpoints page lists your endpoints before the models you deployed from Optimize. For each endpoint it shows the id, model, base URL, status, provider and limits, and the price per million tokens: input, cached input and output, and the cache write where the rate card has one above $0.

What serves your request

1. 01
   Your dedicated capacity first.
2. 02
   Then the shared pool, for the same model only. The request moves on when the dedicated deployment fails before your answer starts, when its circuit is open, or when its capacity is full. It goes to public deployments of the same model that meet your endpoint's constraints, never to a different model.
3. 03
   The commit boundary holds. Once the first token reaches you, the request stays with that provider. See Routing & failover.

Your endpoint's configuration sets the constraints. They apply to every fallback:

• A host that has not stated a value for a constrained property does not qualify.
• A fallback that reaches several hosts through one route sends the constraints with the request. Only hosts that meet them serve it.
• A region constraint excludes such a route, because the route makes no promise about where a host runs.

An endpoint set to primary_only never falls back. When the dedicated deployment cannot serve, you get the error.

Request-level provider fields

For one request you can narrow the routes. You cannot widen them.

• "provider": {"allow_fallbacks": false} keeps the request on your dedicated capacity. It works on open and closed endpoints.
• On an open endpoint, provider.only and provider.ignore filter by provider name, as on any request.
• A closed endpoint selects its providers and hosts through the contract configuration. provider.only, provider.ignore, provider.order, provider.upstream and provider.require_parameters return 400 invalid_request, empty selectors included. Omit those fields on closed endpoints.
• provider.sort and provider.max_price return 400 invalid_request on every dedicated endpoint, open or closed. The message names the field. The endpoint's own constraints, including its maximum price, decide its routes.

Closed endpoints

An endpoint is open unless your contract says otherwise. An open endpoint names the deployment and host of the shared pool that served you, like any request. Your dedicated capacity shows as routerplus.

A closed endpoint never names a deployment or a host. Every surface names RouterPlus as the provider, whichever route served the request. Only the route role differs between routes:

Records stay masked once an endpoint has been closed. Requests it served before it closed, and after it opens again, read as above.

The fields of a closed chat completion:

• Every other field is dropped, including any field a provider adds later.
• A field without a value is left out. A message keeps content: null beside tool calls.
• A model's thinking arrives as reasoning_content on every route.
• A tool call's id is the gateway's own: call_ and 24 hex characters. It is the same on every chunk of the call, and it is the tool_use id on /v1/messages.
• In a stream, the first delta of a choice has the role. The first delta of a tool call has its id, type and name. The finish and the usage each come in a chunk of their own.

Prices

• A closed endpoint bills your rate card on every route. A second price would tell you which route served.
• RouterPlus tries a fallback only when its price is known and at or below the endpoint's maximum price, if one is set. This holds whichever price you then pay.
• In /v1/generation, an attempt billed at your rate card has inference_price_source customer_rate and a price_snapshot_id of the form dedicated:<endpoint id>:r<revision>, which names the rate card that produced the charge.
• usage.cost on the response is the full debit across all attempts, as on every request.

Limits

For the endpoint's traffic, its own limits replace the default organization, workspace, principal and key limits:

A limit your organization set in Billing > Rate limits (/console/limits) still applies on top. You can lower what one key or workspace may use. You cannot raise the endpoint's limits.

A refusal is the usual 429 rate_limit (see Rate limits & spend caps), with x-tm-limit-scope: endpoint:

GET /v1/limits?model=<your-namespace>/<name> returns your key's effective limits on the endpoint and a dedicated object: the endpoint, its revision, the model, the output default, the fallback caps and whether it spills over.

When the shared counter store is unreachable:

• In outage mode closed, requests are refused.
• In the default bounded_open mode, requests are admitted under the endpoint's own limits, divided among the gateways that serve it, for up to 10 minutes. See Limits and capacity.

The paced limit

When traffic arrives at random, a limit that counts each calendar second refuses requests in many seconds, even when the average rate is below the limit. A paced limit queues a short burst instead. It has three settings: the paced rate, a hard rate above it, and the longest wait (the bound).

• Requests leave admission no faster than the paced rate. A request that must wait waits in the gateway. x-tm-queue-ms on the response gives the wait in milliseconds.
• A request whose wait would pass the bound gets 429 with x-tm-limit-kind: rps_queue. Its retry-after says when its wait would fit.
• The hard rate refuses a request at once, with x-tm-limit-kind: rps. It allows a burst as long as the bound (at least one second). It never refuses a request the queue would hold. With a bound of a second or more, a full queue refuses first.
• The wait comes before the request's own deadline starts, so it does not use the endpoint's total time.
• A waiting request holds no concurrency slot. Its place in the pace stays used if you disconnect.

For example, with a paced rate of 6 requests a second, a hard rate of 6.5 and a 2 s bound:

A queue cannot hold a steady excess. Over time, the endpoint passes its paced rate.

Timeouts and defaults

A shorter header wait (streamed) or attempt limit (not streamed) makes failover faster when the dedicated deployment stalls. A stalled request still takes that limit plus the fallback's own time. The attempt limit applies only while another route remains. The last route can run for the rest of the total time.

Other limits

• POST /v1/route does not explain dedicated endpoints yet.
• Every request to a deployment shares its health circuit. Two failures in a row on the dedicated deployment send all its traffic to the fallbacks for 30 seconds. Per-endpoint circuit settings are not available yet.
• RouterPlus changes an endpoint's configuration on request. Each change is a new revision. It reaches every gateway within seconds.