Providers and models

Getting traffic to something that answers it. Most of this section is shorter than it looks, because every endpoint needs three facts and nothing else.

Start by assuming it just works

A provider is a name, a dialect and a URL. Anything that speaks OpenAI’s or Anthropic’s wire format is reachable that way, your own vLLM, a regional endpoint, a vendor nobody has heard of. → Add a provider

Copy a recipe for a vendor you know

Gate ships one complete entry per vendor: the endpoint, the credential reference and the quirks written out. Copy it from the reference page or pick it in the console, then edit it. A recipe is a starting point. The saved document carries the whole entry, nothing in it refers back to the recipe, and the gateway reads no recipe at runtime. → Provider recipes

The distinction is worth holding on to: a provider is an entry you own, and a recipe is one written out for you to copy. There is no catalog the gateway consults and nothing to correct in a release.

Then check that the numbers are the provider’s

The gateway never counts tokens itself. It reads the provider’s own counts off the response, which is why metering works at all. Reconciliation is possible rather than circular, because the ledger can be compared against the bill, and the drift has a metric. → Audit your metering against the provider

Related

Fidelity covers which entry serves a given request when a vendor is declared once per dialect, and what a translation costs when none speaks the request’s dialect.