---
title: Custom Providers
description: Learn how to integrate custom OpenAI-compatible providers with LLMGateway for enhanced flexibility and control.
icon: Columns3Cog
---

import { Callout } from "fumadocs-ui/components/callout";

# Custom Providers

LLMGateway supports integrating custom OpenAI-compatible providers, allowing you to use any API that follows the OpenAI chat completions format. This feature is perfect for:

- Private or self-hosted LLM deployments
- Specialized AI providers not natively supported
- Internal AI services within your organization
- Testing against different model endpoints

<Callout type="info">
	Custom providers must be OpenAI-compatible, supporting the
	`/v1/chat/completions` endpoint format.
</Callout>

## Quick Setup

### 1. Add a Custom Provider Key

Navigate to your organization's provider settings and add a custom provider via the UI.
Provide a lowercase name, OpenAI-compatible base URL, and API token for the custom provider.

### 2. Make Requests

Once configured, make requests using the format `{customName}/{modelName}`:

```bash
curl -X POST "https://api.llmgateway.io/v1/chat/completions" \
  -H "Authorization: Bearer $LLM_GATEWAY_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "mycompany/custom-gpt-4",
    "messages": [
      {
        "role": "user",
        "content": "Hello from my custom provider!"
      }
    ]
  }'
```

## Configuration Requirements

### Custom Provider Name

- **Format**: Lowercase letters (`a-z`) with optional single hyphens between them
- **Examples**: `mycompany`, `internal`, `my-company`, `eu-west`
- **Invalid**: `MyCompany`, `my_company`, `123test`, `-mycompany`, `my-`, `my--company`

<Callout type="warn">
	The custom provider name must match the regex pattern `/^[a-z]+(-[a-z]+)*$/`
	exactly.
</Callout>

### Base URL

- Must be a valid URL pointing to your provider's base endpoint
- Use an `https://` URL for hosted providers; an `http://` URL is allowed when
  self-hosting (for example a service on your own network)
- LLMGateway will append `/v1/chat/completions` automatically
- **Example**: `https://api.example.com` → `https://api.example.com/v1/chat/completions`

### API Token

- Provider-specific authentication token
- Used in the `Authorization: Bearer {token}` header

<Callout type="info">
	Unlike built-in providers, custom provider models are not validated, giving
	you complete flexibility.
</Callout>

## Supported Features

Custom providers inherit full LLMGateway functionality.

## Custom Model Catalog

<Callout type="info">
	The custom model catalog is an **Enterprise** feature. Managing entries
	requires an enterprise plan, but the gateway always honors catalog entries
	that already exist.
</Callout>

By default, requests through a custom provider are **not billed** and have **no
enforced limits** — LLMGateway has no catalog entry for the model, so it cannot
know its pricing, context window, or capabilities. The **custom model catalog**
lets you define that information per custom provider key, so LLMGateway can
attribute cost, enforce limits, and report usage just like a built-in model.

Custom catalog models are **text-output** models. Multi-modal _input_ (images,
audio) is still supported and can be priced via the input fields below; output
generation (images, video, audio) is intentionally out of scope because it is
too provider-specific to bill generically.

### Defining a model

Open **Organization → Models**, pick the custom provider key, and add a
catalog entry. Every field except the model id is optional:

| Field                    | Purpose                                                                                                                         |
| ------------------------ | ------------------------------------------------------------------------------------------------------------------------------- |
| **Model id**             | The id used after the provider prefix (e.g. `gpt-5.5` in `mycompany/gpt-5.5`).                                                  |
| **Display name**         | Optional human-readable label.                                                                                                  |
| **Context size**         | Maximum input window in tokens. Enforced per request.                                                                           |
| **Max output**           | Maximum completion tokens. Enforced against `max_tokens`.                                                                       |
| **Token prices**         | Input, output, cached input, cache read, cache write (5m and 1h), per-request, web search, image input, and audio input prices. |
| **Capabilities**         | `streaming`, `vision`, `tools`, `reasoning`, `jsonOutput`, `audio`.                                                             |
| **Supported parameters** | Advisory list of accepted request parameters.                                                                                   |

Prices are in **USD per token** and accept either decimal (`0.000003`) or
exponent (`3.0e-6`) notation.

### Cost attribution

When a request matches a catalog entry, LLMGateway bills it using the entry's
token prices and records the cost on the activity log. Without a matching entry
the request stays unbilled (zero/null cost).

<Callout type="warn">
	A known model id (for example `gpt-5.5`) routed through a custom provider is
	**not** billed at that model's public pricing — only the prices you define in
	the catalog apply. Define the model in the catalog to attribute cost.
</Callout>

### Limit and capability enforcement

When an entry defines limits or capabilities, LLMGateway enforces them before
forwarding the request:

- **Context size / max output** — requests that would exceed the configured
  window or output budget are rejected with `400`.
- **Capabilities** — when a flag is explicitly set to disabled, requests that use
  that feature are rejected (e.g. images against a non-`vision` model, tools
  against a non-`tools` model, streaming against a non-streaming model). Flags
  left unset stay permissive and the upstream provider enforces them.

### Restricting to the catalog

Each custom provider key has an **Only allow catalog models** switch. When
enabled, requests through that provider must reference a defined catalog model —
undefined models are rejected with `400`. This guarantees that every request has
known cost attribution and enforced limits. When disabled, undefined models
still work but remain unbilled, as before.

## Visibility

Every active organization member — including project-scoped **developer**
members — can browse the org's custom providers and their model catalogs
read-only on the **Models** page, so developers can always discover
which providers and models are available to them. Creating, editing, and
deleting providers, models, and attestations stays restricted to organization
owners and admins.

## Compliance attestation

If your organization enforces a [provider compliance policy](/features/compliance),
custom providers are blocked by default — they have no catalogue data policy, so
the fail-closed rules treat them as non-compliant. Organization owners and
admins can record a per-key **self-attestation** of the deployment's compliance
posture under **Models → Compliance attestation**, which the gateway then
evaluates against the policy exactly like a catalogue provider's data policy.
See [Compliance → Custom providers](/features/compliance#custom-providers) for
details and caveats.
