> ## Documentation Index
> Fetch the complete documentation index at: https://docs.openlayer.com/llms.txt
> Use this file to discover all available pages before exploring further.

# Context relevancy

> Learn how to use the context relevancy test

## Definition

The context relevancy test measures how relevant the context retrieved is given the question. This metric is based on the Ragas [context precision](https://docs.ragas.io/en/stable/concepts/metrics/available_metrics/context_precision/) metric.

## Taxonomy

* **Task types**: LLM.
* **Availability**: development (Continuously evaluate your models and datasets as you iterate on their versions.)
  and monitoring (Monitor a model in production, measure its health, check for drifts and set up alerts.).

## Why it matters

* Context relevancy ensures that your retrieval system provides information that is directly related to the user's question.
* This metric helps identify when your retrieval mechanism is returning irrelevant or off-topic context that could confuse the LLM.
* It's essential for RAG (Retrieval-Augmented Generation) systems to maintain high precision in retrieved information.

## Required columns

To compute this metric, your dataset must contain the following columns:

* **Input**: The question or prompt given to the LLM
* **Ground truth**: The reference/correct answer
* **Context**: The retrieved context or background information

Without an explicit question column, Openlayer scores the input variable that most looks like the question and skips the metric if no input holds text. See [Which input is the question](https://docs.openlayer.com/integrations/ragas.md#which-input-is-the-question).

> [!NOTE]
>
> This metric relies on an LLM evaluator judging your submission. On Openlayer,
> you can configure the underlying LLM used to compute it. Check out the
> [OpenAI](https://docs.openlayer.com/integrations/openai.md#openai-llm-evaluator) or
> [Anthropic](https://docs.openlayer.com/integrations/anthropic.md#anthropic-llm-evaluator) integration
> guides for details.

## Test configuration examples

If you are writing a `tests.json`, here are a few valid configurations for the context relevancy test:

```json Development theme={null}
[
  {
    "name": "Context relevancy above 0.8",
    "description": "Ensure that retrieved context is highly relevant to the question with a score above 0.8",
    "type": "performance",
    "subtype": "metricThreshold",
    "thresholds": [
      {
        "insightName": "metrics",
        "measurement": "contextRelevancy",
        "operator": ">",
        "value": 0.8
      }
    ],
    "subpopulationFilters": null,
    "mode": "development",
    "usesValidationDataset": true,
    "usesTrainingDataset": false,
    "usesMlModel": true,
    "syncId": "b4dee7dc-4f15-48ca-a282-63e2c04e0689"
  }
]
```

```json Monitoring theme={null}
[
  {
    "name": "Context relevancy above 0.8",
    "description": "Ensure that retrieved context is highly relevant to the question with a score above 0.8",
    "type": "performance",
    "subtype": "metricThreshold",
    "thresholds": [
      {
        "insightName": "metrics",
        "measurement": "contextRelevancy",
        "operator": ">",
        "value": 0.8
      }
    ],
    "subpopulationFilters": null,
    "mode": "monitoring",
    "usesProductionData": true,
    "evaluationWindow": 3600,
    "delayWindow": 0,
    "syncId": "b4dee7dc-4f15-48ca-a282-63e2c04e0689"
  }
]
```

## Related

* [Ragas integration](https://docs.openlayer.com/integrations/ragas.md) - Learn more about Ragas metrics.
* [Context recall test](https://docs.openlayer.com/tests/catalog/context-recall.md) - Measure completeness of retrieved context.
* [Context utilization test](https://docs.openlayer.com/tests/catalog/context-utilization.md) - Evaluate how well context is used.
* [Aggregate metrics](https://docs.openlayer.com/tests/performance/aggregate-metrics) - Overview of all available metrics.


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.