---
title: Service Tiers
description: Trade latency against cost on supported OpenAI, Google, and Fireworks models with Flex and Priority processing tiers.
icon: Gauge
---

import { Callout } from "fumadocs-ui/components/callout";

# Service Tiers

Some OpenAI, Google, and Fireworks models support selectable **processing tiers** that trade
latency and availability against price. You pick one per request with the
OpenAI-compatible `service_tier` parameter, and LLM Gateway forwards it only
when the selected provider/model mapping supports that tier.

| Tier         | `service_tier`            | Cost vs. standard | Latency / availability                      |
| ------------ | ------------------------- | ----------------- | ------------------------------------------- |
| Standard     | `default` / `auto` / omit | baseline          | Normal on-demand latency                    |
| **Flex**     | `flex`                    | **−50%**          | Best-effort; may be preempted under load    |
| **Priority** | `priority`                | varies by model   | Prioritized above standard and flex traffic |

## Using the `service_tier` parameter

```bash
curl -X POST "https://api.llmgateway.io/v1/chat/completions" \
  -H "Authorization: Bearer $LLM_GATEWAY_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "google-vertex/gemini-3.1-pro-preview",
    "service_tier": "priority",
    "messages": [
      { "role": "user", "content": "Summarize this incident report." }
    ]
  }'
```

Accepted values are `flex`, `priority`, and `default`/`auto` (standard). If you
request `flex` or `priority` for a provider/model mapping that does not support
that tier, the gateway returns a 400 `unsupported_service_tier` error and logs
the request as a client error.

The parameter works the same on the OpenAI-compatible **Responses API**
(`/v1/responses`): the tier is forwarded to the provider and the response's
`service_tier` field echoes the tier that was actually served.

## Supported providers

Service tiers are explicit per provider/model mapping. Check the model page for
the exact tiers exposed by each provider card.

- **OpenAI** (`openai`) — sent as the OpenAI `service_tier` request field for
  supported OpenAI models. Flex is billed at 0.5x standard token prices and
  Priority uses the model-specific multiplier shown on the model page.

- **Google Vertex AI** (`google-vertex`) — sent as the
  `X-Vertex-AI-LLM-Shared-Request-Type` request header, together with
  `X-Vertex-AI-LLM-Request-Type: shared` so the request bypasses any Provisioned
  Throughput on the project and actually reaches the shared Flex/Priority tier.
  Flex and Priority are served only on the **global** endpoint, which is the
  gateway default. Google Flex PayGo applies a 0.5x multiplier; Google Priority
  PayGo applies a 1.8x multiplier.
- **Google AI Studio / Gemini API** (`google-ai-studio`) — sent as a
  `service_tier` field in the request body for configured models that opt in.

- **Fireworks AI** (`fireworks`) — sent as a `service_tier` field in the request
  body on its OpenAI-compatible chat completions endpoint. Only Priority is
  offered (Fireworks publishes no Flex rate card), billed at a 1.25x multiplier.
  Fireworks does not report the tier it served, but a Priority request is either
  served at Priority or shed with a 503, so an accepted request is billed at the
  tier it was sent at.

<Callout type="info">
	Tiers are supported on a **subset** of models, and the Flex and Priority
	subsets differ by provider. For example, Google Flex PayGo lists Gemini 3
	image / Nano Banana models, but Google Priority PayGo does not; those
	configured image mappings are Flex-only.
</Callout>

<Callout type="warn">
	Flex and Priority are only honored when the request reaches the provider
	directly, so a provider key with a **custom base URL** (a proxy) is excluded
	from service-tier routing — a proxy may silently drop the tier and serve
	standard. This applies to every provider that offers tiers, including OpenAI:
	the tier travels as a `service_tier` body field that an OpenAI-compatible
	proxy is free to ignore. With multiple providers/keys, the gateway routes
	around the ineligible key automatically; if a request pins a provider whose
	only key uses a custom base URL, it returns a 400 instead of silently
	downgrading. Keys with no custom base URL (the managed default) are always
	eligible.
</Callout>

## Retries and fallback never downgrade the tier

A request can change provider or credential mid-flight: LLM Gateway falls back
to another provider when one fails, and rotates to another key for the same
provider when a credential returns a 429 or an auth error. A requested tier is
carried through all of it.

- Provider routing is narrowed to mappings that support the requested tier
  **before** a provider is picked, so no fallback candidate can be one that
  would serve the request as standard.
- Key selection — BYOK keys, platform-managed credentials, and env credentials
  alike — skips any credential that cannot carry the tier (a proxy base URL, or
  a Vertex credential pinned to a regional endpoint), on the first attempt and
  on every retry.
- Every attempt re-resolves the tier against the provider, region and
  credential it actually resolved to, and fails rather than sending at a lower
  tier. If no eligible candidate is left, you get the upstream error instead of
  a silently downgraded response.

This applies to a tier you requested yourself. The optional coding-plan default
tier is a cost preference rather than a requirement, so a request that cannot be
served at that tier runs at standard instead of failing; `used_service_tier` in
the response metadata always reports what was actually served.

<Callout type="info">
	Rotating to another key for the same provider is a separate upstream account
	as far as prompt caching is concerned, so a retried request re-writes its
	cached prefix rather than reading the original one. Send `x-no-fallback: true`
	to have the original upstream error returned to you — note that this disables
	cross-provider fallback, not key rotation within a provider.
</Callout>

## Pricing uses multipliers

Service tiers do not define separate model prices in LLM Gateway. They multiply
the provider mapping's standard token prices:

- Standard / `default` / `auto`: 1x
- Flex: 0.5x
- Priority: model/provider-specific, shown on the model page

The multiplier scales per-token costs, including input, output, cached, and
image tokens. Flat per-request and web-search fees are not tier-scaled.

## Billing follows the served tier

When a provider reports the tier that was actually served, LLM Gateway bills
that returned tier instead of blindly billing the requested value:

- A `priority` request that runs as priority is billed at 2.5x.
- A `flex` request that runs as flex is billed at 0.5x.
- A request that is served as standard is billed at the standard 1x rate.

The served tier is read back from the provider response — Vertex reports it in
`usageMetadata.trafficType` (`ON_DEMAND_PRIORITY` / `ON_DEMAND_FLEX` /
`ON_DEMAND`), Google AI Studio reports it in the `x-gemini-service-tier`
response header, and OpenAI can return `service_tier` in response payloads or
stream events. Providers that report no tier at all (Fireworks) never downgrade
silently — they reject the request instead — so an accepted request is billed at
the tier it was sent at.

<Callout type="warn">
	LLM Gateway rejects unsupported tier requests before provider routing. For
	example, `gemini-3-pro-image-preview` currently exposes Flex for Google AI
	Studio and Vertex, but not Priority.
</Callout>

You can see per-tier pricing for each model on its
[model page](https://llmgateway.io/models). Supported provider cards include a
Service Tier selector in the card header and show the active multiplier next to
each tier.

## Sources

- [OpenAI API pricing](https://openai.com/api/pricing/)
- [Google Flex PayGo](https://docs.cloud.google.com/vertex-ai/generative-ai/docs/flex-paygo)
- [Google Priority PayGo](https://docs.cloud.google.com/gemini-enterprise-agent-platform/models/priority-paygo)
- [Fireworks Serverless Priority and Fast](https://docs.fireworks.ai/serverless/priority-and-fast)
