Environment Variables

Learn where Openlayer uses workspace and project environment variables, how your code and LLM judges read them, and how to manage them.

Environment variables are name-value pairs, such as API keys, endpoints, and other configuration, that you store in Openlayer at the workspace or project level. Openlayer passes them to the code and LLM judges it runs on your behalf, so you never put credentials in your repository or in openlayer.json.

​

Where environment variables are used

Where	Variables it reads	Project overrides workspace
Development mode runs	All of them: pre-defined and custom, secret and plain	Yes
Custom metrics	All of them: pre-defined and custom, secret and plain	Yes
LLM-based tests, such as LLM-as-a-judge	The pre-defined variables of the judge’s provider	Yes
Test and session summaries	The pre-defined variables of the judge’s provider	Yes
Jev-as-a-judge	TYPESAFE_API_KEY	Yes
Custom CA certificates	Any variable named CUSTOM_CA_CERT_*, for all of the surfaces above	Yes

LLM-based tests, Jev-as-a-judge, and custom metrics run on Openlayer in both development and monitoring mode, so they read these variables in either mode.

​

What doesn’t read them

• Tracing in your own application. When you instrument your code for monitoring, the Openlayer SDK runs in your process and reads your own environment, such as OPENLAYER_API_KEY and OPENLAYER_INFERENCE_PIPELINE_ID. Set those, and your LLM provider keys, where your application runs. Openlayer’s environment variables don’t reach it.
• Code that runs on your machine or in your CI. The CLI uses your local credentials when you push. If you generate your model outputs before pushing, your batchCommand doesn’t run on Openlayer, so it never sees these variables.

​

How to use them

​

In development mode runs

When Openlayer runs your AI system on your datasets (see Configuring output generation), the installCommand and batchCommand from your openlayer.json run with every workspace and project variable set as a process environment variable, under the name you gave it. Read them the way you would read any environment variable:

Python

import os

from openai import OpenAI

# The OpenAI client reads OPENAI_API_KEY from the environment.
client = OpenAI()

# A custom variable you added under Settings → Environment.
retriever_url = os.environ["RETRIEVER_URL"]

Keep in mind:

• Openlayer only runs these commands when the commit doesn’t already include your model outputs. See How Openlayer checks if it should compute outputs.
• Openlayer reads the variables when each run starts, so a change applies from the next commit.
• Openlayer sets OPENLAYER_API_KEY, OPENLAYER_PROJECT_ID, and OPENLAYER_WORKSPACE_ID in the run for you, so you don’t need to store them.
• During the batchCommand, the SDK’s tracing doesn’t publish to a data source, so traced functions in your run script don’t send data to monitoring.

​

In custom metrics

The installCommand and runCommand in a custom metric’s config.json run with the same variables as a development mode run. Read them from os.environ in your metric’s run.py. This applies when the metric runs on a development commit and on monitoring data.

​

In LLM-based tests

LLM-based tests, such as LLM-as-a-judge, call the judge model you select in your project’s Settings → LLM-as-a-judge. The judge authenticates with the pre-defined variables of that model’s provider:

• OpenAI — OPENAI_API_KEY. To send requests to an OpenAI-compatible endpoint, also set OPENAI_BASE_URL, and set OPENAI_CUSTOM_HEADERS to comma-separated Name:value pairs for any extra headers. See Using OpenAI LLMs as the LLM judge.
• Azure OpenAI — AZURE_OPENAI_ENDPOINT, plus either AZURE_OPENAI_API_KEY or, for Microsoft Entra ID, all of MSAL_TENANT_ID, MSAL_CLIENT_ID, MSAL_CLIENT_SECRET, and MSAL_SCOPES.
• Anthropic — ANTHROPIC_API_KEY. See Using Anthropic LLMs as the LLM judge.
• AWS Bedrock — AWS_REGION, plus either AWS_BEARER_TOKEN_BEDROCK, or AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY. Add AWS_BEDROCK_ROLE_ARN to have those keys assume a role with Bedrock access. See Using Bedrock LLMs as the LLM judge.
• Google — GOOGLE_API_KEY for Google AI Studio, or, for Vertex AI, GOOGLE_GENAI_USE_VERTEXAI set to true and GOOGLE_CLOUD_PROJECT. GOOGLE_CLOUD_LOCATION defaults to us-central1, and GOOGLE_APPLICATION_CREDENTIALS_JSON holds a service account key. See Using Google Gemini models as the LLM judge.
• Cohere — COHERE_API_KEY.

If a required variable is missing, Openlayer skips the test and the result names the missing variables. The Groq and Mistral pre-defined variables aren’t used by LLM judges. They reach only your development mode runs and custom metrics.

​

In test and session summaries

Test summaries and session summaries use the project’s LLM judge, so they read the same provider variables as LLM-based tests. Without them, Openlayer doesn’t generate the summary.

​

In Jev-as-a-judge

Jev-as-a-judge reads TYPESAFE_API_KEY, listed under Other in the pre-defined variables. Openlayer resolves the key from the project, then the workspace, then a platform key. A platform key still runs the test, and the rows are judged through Openlayer’s TypeSafe account. If no key resolves, the test is skipped.

​

Managing environment variables

​

Workspace-level environment variables

Workspace-level environment variables are shared across all projects in your workspace.

To set them, navigate to Settings → Environment. The page has three tabs: Pre-defined variables, Secret variables, and Plain variables.

​

Pre-defined variables

Openlayer provides pre-defined fields for the variables listed in LLM-based tests, grouped by provider (OpenAI, Azure OpenAI, Anthropic, AWS Bedrock, Cohere, Google, Groq, Mistral, and Other). Each group links to that provider’s page on where to find the keys.

​

Secret and plain variables

Beyond the pre-defined fields you can add your own variables, in two kinds:

Kind	Stored	Visible to workspace members	Use for
Secret	Encrypted at rest	No, the value stays masked	API keys, tokens, passwords
Plain	As entered	Yes, the value is readable	Client IDs, regions, endpoints, other configuration

New variables are secret unless you add them under Plain variables. A secret value can be replaced but never read back, so use plain for anything a teammate may legitimately need to look up, and keep credentials secret. Both kinds reach your code the same way.

​

Project-level environment variables

Project-level environment variables work the same way as workspace-level ones, but are specific to a single project.

To set them, open your project and navigate to Settings → Environment. The page has the same three tabs as the workspace page.

When the project and the workspace set a variable with the same name, the project value wins everywhere in that project. On the project page, a pre-defined variable with no project value but a workspace value shows Inherited from workspace. One that has its own project value while the workspace also sets it shows Overriding workspace.

​

Custom CA certificates

If your organization uses custom Certificate Authority (CA) certificates for secure connections, for example to a model behind an internal gateway, add each certificate as a variable whose name starts with CUSTOM_CA_CERT_, followed by a descriptive suffix (for example, CUSTOM_CA_CERT_INTERNAL). Paste the certificate in PEM format as the value. Add it on the workspace page, or on the project page for a single project.

Openlayer trusts these certificates in:

• Development mode runs and custom metrics — they’re installed in the run environment before your commands start.
• LLM judges — they’re trusted for the judge’s connections to your LLM provider.

You can add multiple certificates with different suffixes (for example, CUSTOM_CA_CERT_1, CUSTOM_CA_CERT_2). All variables matching CUSTOM_CA_CERT_* are trusted.

​

Verifying a provider key

The Environment page stores keys but doesn’t verify them. To check that a key reaches its provider, open your project and go to Settings → LLM-as-a-judge, which holds the defaults for LLM-based tests:

• Default LLM judge — the default LLM for computing LLM-based metrics.
• Default LLM — a picker for the judge model, such as OpenAI / gpt-4.1-mini-2025-04-14. Choose one of the predefined models it lists, or type a model name and choose Select.
• Test connection — checks that Openlayer can reach the selected model.

As the page notes, you add provider API keys via the workspace environment or the project environment, so if the connection fails, review the matching key under Settings → Environment.