> ## Documentation Index
> Fetch the complete documentation index at: https://docs.openlayer.com/llms.txt
> Use this file to discover all available pages before exploring further.

# Overview

> Route your LLM traffic through one endpoint to get spend controls, key management, guardrails, and observability, without changing your app code

The **Openlayer Gateway** is a single endpoint that sits between your apps and your LLM providers
and **enforces your rules on every request, in real time**. It blocks calls that go over budget and,
on supported content endpoints, stops prompt injection and redacts PII before a request reaches a
provider or a response reaches a user.

Every request that flows through the gateway is also [traced to your Openlayer project](https://docs.openlayer.com/gateway/observability.md), so
the monitoring you already rely on keeps working.

## How it works

**1. Point your app at the gateway**

Keep your existing OpenAI- or Anthropic-compatible client. Point its base URL at the gateway
and swap in a gateway key.

For example:

```python theme={null}
from openai import OpenAI

client = OpenAI(
    base_url="https://your-gateway.example.com/v1",  # the gateway, not api.openai.com
    api_key="sk-olga-...", # a gateway key, not your provider key
)

response = client.responses.create(model="gpt-4o-mini", input="Hello!")
```

**2. Enforce your rules**

The gateway verifies the key, then checks the request in the order that costs least first:
[model access](https://docs.openlayer.com/gateway/model-access.md) (`403`), then
[usage limits](https://docs.openlayer.com/gateway/budgets-and-limits.md) (`429`), then, when the endpoint supports them, your
[content guardrails](https://docs.openlayer.com/gateway/guardrails.md) (`422`). Anything refused is blocked here, before it
reaches a provider.

**3. Route to a provider**

It forwards the request to the provider you configured, and can override the model or
**translate text-generation requests between formats**, so an OpenAI Responses request runs
against an Anthropic model, and vice versa.

**4. Return and record**

On supported content endpoints, [output guardrails](https://docs.openlayer.com/gateway/guardrails.md) run on the way back,
then the response streams to your app. Every request is also published as a
[trace](https://docs.openlayer.com/gateway/observability.md) in your Openlayer project, fire-and-forget, so tracing never
slows or breaks a response.

![Every gateway request becomes a trace in Openlayer](https://mintcdn.com/openlayer-44/2eGfXTpwMAOWCnGg/images/gateway/gateway_trace.png?fit=max\&auto=format\&n=2eGfXTpwMAOWCnGg\&q=85\&s=9bdc0c30431f87899e5c79d81a7318df)

## What you get

Everything is configured once, in the gateway's admin portal. There are no redeploys and no code
changes in the apps sending traffic. The portal opens on the **Dashboard**, where you can see what
the whole deployment is spending before going into any one policy.

![The Openlayer Gateway admin portal, open on the Dashboard](https://mintcdn.com/openlayer-44/c_6-GM8DMEh-Ygkc/images/gateway/dashboard.png?fit=max\&auto=format\&n=c_6-GM8DMEh-Ygkc\&q=85\&s=d30a25efe0c388d52985b82b8688a239)

> [card] [Access & keys](https://docs.openlayer.com/gateway/api-keys.md)
>
> Issue and revoke keys for each app or teammate, assign them to groups, and
> hand out self-serve invite links.

> [card] [Budgets & usage](https://docs.openlayer.com/gateway/budgets-and-limits.md)
>
> Cap spend, requests, or tokens per key, user, or group, and see exactly
> what's being consumed and what it costs.

> [card] [Guardrails](https://docs.openlayer.com/gateway/guardrails.md)
>
> Block or redact PII, stop prompt injection, and screen for unsafe content on
> inputs and outputs, before they reach a provider or a user.

> [card] [Model access](https://docs.openlayer.com/gateway/model-access.md)
>
> Allow or block models per key, user, group, or the whole deployment.

> [card] [Providers & models](https://docs.openlayer.com/gateway/connect-providers.md)
>
> Connect OpenAI, Anthropic, and Azure upstreams, route traffic between them,
> and call any model from any SDK.

## FAQ

> [accordion] Do I have to change my application code?
>
> Almost nothing. You point your client at the gateway's base URL and use a gateway key instead of
> a provider key. That's it. Clients that speak the **OpenAI Responses API**, **OpenAI Embeddings
> API**, or **Anthropic Messages API** work unchanged. Streaming works on the text-generation
> endpoints.

> [accordion] How is this different from instrumenting my code with the Openlayer SDK?
>
> They serve different jobs and you can use either or both.
>
> The **gateway** exists to enforce your rules on every request, in real time. Tracing comes as a
> byproduct: because traffic flows through the gateway, every request is also
> [published to your Openlayer project](https://docs.openlayer.com/gateway/observability.md) without any SDK in your apps. That
> makes it the right path when you want governance plus baseline observability across many apps or
> languages at one chokepoint.
>
> The **[Openlayer SDK](https://docs.openlayer.com/monitoring/instrument.md)** runs inside your code. It doesn't enforce anything
> at the request boundary, but it captures detail the gateway can't see from outside: nested spans
> for retrieval, tools, and sub-calls in a multi-step pipeline, plus any custom metadata. Reach for
> it when you need fine-grained traces of how a feature actually runs.
>
> A common setup is the gateway for governance and baseline traces, plus the SDK in the places that
> need the deeper view.

> [accordion] Is my prompt and response data stored in the gateway?
>
> The gateway forwards traffic. It doesn't keep your prompts and responses. It stores hashed API
> keys, per-day usage tallies, and guardrail event metadata (which guardrail triggered and what
> kind of entity, not the underlying text).
>
> The full request and response content is sent to **your Openlayer project** as a trace, where
> your data-retention and access controls apply. Openlayer can run on-premises or as
> SOC 2 Type II compliant SaaS.

> [accordion] Can I use OpenAI models from the Anthropic SDK, or vice versa?
>
> Yes. When you route a request to a provider whose format differs from your client's, the gateway
> translates the request, response, and stream between the OpenAI Responses API and the Anthropic
> Messages API. See [Use any model from any SDK](https://docs.openlayer.com/gateway/use-any-model.md).

> [accordion] Which providers are supported?
>
> OpenAI, Anthropic, and Azure OpenAI / Foundry today. Any provider that exposes an
> OpenAI- or Anthropic-compatible API can be connected. See
> [Connect providers](https://docs.openlayer.com/gateway/connect-providers.md).

> [accordion] What happens if the gateway is sent more traffic than it can handle?
>
> The gateway caps how many requests it works on at once and keeps a shallow queue for bursts.
> Past that, it turns requests away with **503** and a `Retry-After` header instead of degrading
> for everyone, so a traffic spike costs you the requests it sheds rather than the deployment.
>
> This is a capacity backstop, not a policy: it sits well above normal throughput, and it's
> separate from the `429` a [usage limit](https://docs.openlayer.com/gateway/budgets-and-limits.md) returns. Retrying with
> backoff is the right response. The admin portal stays available while it's happening.


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.