# explodinggradients/ragas architecture

> Architecture diagram of the GitHub repository explodinggradients/ragas, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-09-30). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/explodinggradients/ragas
- Repository: https://github.com/explodinggradients/ragas

## Overview

Ragas evaluates LLM applications with built-in LLM-based and traditional metrics, and can generate test data when none exists. Its runtime also provides model adapters, dataset handling, framework/observability integrations, and optimization-related components. The principal workflow is documented in the README; generation and several internal dependencies are supported by indexed source references. Unsampled evaluator, backend, and integration internals limit relationship detail, so uncertain links are omitted.

Main entry points: [`src/ragas/testset/synthesizers/generate.py`](https://github.com/explodinggradients/ragas/blob/main/src/ragas/testset/synthesizers/generate.py) (Test-set generation), [`src/ragas/metrics/base.py`](https://github.com/explodinggradients/ragas/blob/main/src/ragas/metrics/base.py) (Metric execution core), [`src/ragas/llms/base.py`](https://github.com/explodinggradients/ragas/blob/main/src/ragas/llms/base.py) (LLM abstraction), [`src/ragas/metrics/collections`](https://github.com/explodinggradients/ragas/tree/main/src/ragas/metrics/collections) (Evaluation metrics)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_evaluation["Evaluation"]
  node_evaluation_api["Evaluation workflow<br/>[evaluation.py]"]
  node_metrics["Evaluation metrics"]
  node_metric_core["Metric execution core<br/>[base.py]"]
  node_metric_result["Metric results<br/>[result.py]"]
  node_prompts["Evaluation prompts"]
end

subgraph group_generation["Test generation"]
  node_testset_generator["Test-set generation<br/>[generate.py]"]
  node_knowledge_graph["Knowledge graph<br/>[graph.py]"]
  node_graph_transforms["Graph transforms"]
  node_synthesizers["Question synthesizers"]
end

subgraph group_models["Model components"]
  node_llm_api["LLM abstraction<br/>[base.py]"]
  node_litellm_llm["LiteLLM provider<br/>[litellm_llm.py]"]
  node_embeddings["Embedding providers<br/>[base.py]"]
end

subgraph group_data["Data and results"]
  node_dataset["Evaluation datasets<br/>[dataset_schema.py]"]
  node_storage_backends["Dataset backends"]
  node_cache[("Model cache<br/>[cache.py]")]
end

subgraph group_ecosystem["Integrations and improvement"]
  node_provider_integrations["Framework integrations"]
  node_experiment["Production feedback<br/>[experiment.py]"]
  node_optimizers["Evaluation optimizers"]
end

node_developer(("LLM app developer"))
node_app["LLM application"]

node_developer -.->|"starts evaluation"| node_evaluation_api
node_app -.->|"provides outputs"| node_evaluation_api
node_evaluation_api -.->|"evaluates with"| node_metrics
node_metrics -->|"uses scoring core"| node_metric_core
node_metric_core -->|"uses LLM"| node_llm_api
node_metric_core -->|"uses embeddings"| node_embeddings
node_metric_core -->|"uses prompts"| node_prompts
node_metric_core -->|"scores samples"| node_dataset
node_metrics -->|"returns scores"| node_metric_result
node_developer -.->|"requests test data"| node_testset_generator
node_testset_generator -->|"uses graph"| node_knowledge_graph
node_testset_generator -->|"applies transforms"| node_graph_transforms
node_testset_generator -->|"generates samples"| node_synthesizers
node_testset_generator -->|"uses LLM"| node_llm_api
node_testset_generator -->|"uses embeddings"| node_embeddings
node_testset_generator -->|"produces dataset"| node_dataset
node_llm_api -->|"caches requests"| node_cache
node_embeddings -->|"caches embeddings"| node_cache
node_litellm_llm -->|"implements abstraction"| node_llm_api
node_dataset -.->|"exports and loads"| node_storage_backends
node_provider_integrations -.->|"supports feedback loops"| node_experiment
node_developer -.->|"improves evaluations"| node_optimizers

click node_evaluation_api "https://github.com/explodinggradients/ragas/blob/main/src/ragas/evaluation.py"
click node_metrics "https://github.com/explodinggradients/ragas/tree/main/src/ragas/metrics/collections"
click node_metric_core "https://github.com/explodinggradients/ragas/blob/main/src/ragas/metrics/base.py"
click node_metric_result "https://github.com/explodinggradients/ragas/blob/main/src/ragas/metrics/result.py"
click node_prompts "https://github.com/explodinggradients/ragas/tree/main/src/ragas/prompt"
click node_testset_generator "https://github.com/explodinggradients/ragas/blob/main/src/ragas/testset/synthesizers/generate.py"
click node_knowledge_graph "https://github.com/explodinggradients/ragas/blob/main/src/ragas/testset/graph.py"
click node_graph_transforms "https://github.com/explodinggradients/ragas/tree/main/src/ragas/testset/transforms"
click node_synthesizers "https://github.com/explodinggradients/ragas/tree/main/src/ragas/testset/synthesizers"
click node_dataset "https://github.com/explodinggradients/ragas/blob/main/src/ragas/dataset_schema.py"
click node_storage_backends "https://github.com/explodinggradients/ragas/tree/main/src/ragas/backends"
click node_cache "https://github.com/explodinggradients/ragas/blob/main/src/ragas/cache.py"
click node_llm_api "https://github.com/explodinggradients/ragas/blob/main/src/ragas/llms/base.py"
click node_litellm_llm "https://github.com/explodinggradients/ragas/blob/main/src/ragas/llms/litellm_llm.py"
click node_embeddings "https://github.com/explodinggradients/ragas/blob/main/src/ragas/embeddings/base.py"
click node_provider_integrations "https://github.com/explodinggradients/ragas/tree/main/src/ragas/integrations"
click node_experiment "https://github.com/explodinggradients/ragas/blob/main/src/ragas/experiment.py"
click node_optimizers "https://github.com/explodinggradients/ragas/tree/main/src/ragas/optimizers"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_evaluation_api,node_metrics,node_metric_core,node_metric_result,node_prompts toneBlue
class node_testset_generator,node_knowledge_graph,node_graph_transforms,node_synthesizers toneAmber
class node_llm_api,node_litellm_llm,node_embeddings toneMint
class node_dataset,node_storage_backends,node_cache toneRose
class node_provider_integrations,node_experiment,node_optimizers,node_developer,node_app toneIndigo
```

## Components

### Evaluation

- [Evaluation workflow](https://github.com/explodinggradients/ragas/blob/main/src/ragas/evaluation.py): `src/ragas/evaluation.py`
- [Evaluation metrics](https://github.com/explodinggradients/ragas/tree/main/src/ragas/metrics/collections): `src/ragas/metrics/collections`
- [Metric execution core](https://github.com/explodinggradients/ragas/blob/main/src/ragas/metrics/base.py): `src/ragas/metrics/base.py`
- [Metric results](https://github.com/explodinggradients/ragas/blob/main/src/ragas/metrics/result.py): `src/ragas/metrics/result.py`
- [Evaluation prompts](https://github.com/explodinggradients/ragas/tree/main/src/ragas/prompt): `src/ragas/prompt`

### Test generation

- [Test-set generation](https://github.com/explodinggradients/ragas/blob/main/src/ragas/testset/synthesizers/generate.py): `src/ragas/testset/synthesizers/generate.py`
- [Knowledge graph](https://github.com/explodinggradients/ragas/blob/main/src/ragas/testset/graph.py): `src/ragas/testset/graph.py`
- [Graph transforms](https://github.com/explodinggradients/ragas/tree/main/src/ragas/testset/transforms): `src/ragas/testset/transforms`
- [Question synthesizers](https://github.com/explodinggradients/ragas/tree/main/src/ragas/testset/synthesizers): `src/ragas/testset/synthesizers`

### Model components

- [LLM abstraction](https://github.com/explodinggradients/ragas/blob/main/src/ragas/llms/base.py): `src/ragas/llms/base.py`
- [LiteLLM provider](https://github.com/explodinggradients/ragas/blob/main/src/ragas/llms/litellm_llm.py): `src/ragas/llms/litellm_llm.py`
- [Embedding providers](https://github.com/explodinggradients/ragas/blob/main/src/ragas/embeddings/base.py): `src/ragas/embeddings/base.py`

### Data and results

- [Evaluation datasets](https://github.com/explodinggradients/ragas/blob/main/src/ragas/dataset_schema.py): `src/ragas/dataset_schema.py`
- [Dataset backends](https://github.com/explodinggradients/ragas/tree/main/src/ragas/backends): `src/ragas/backends`
- [Model cache](https://github.com/explodinggradients/ragas/blob/main/src/ragas/cache.py): `src/ragas/cache.py`

### Integrations and improvement

- [Framework integrations](https://github.com/explodinggradients/ragas/tree/main/src/ragas/integrations): `src/ragas/integrations`
- [Production feedback](https://github.com/explodinggradients/ragas/blob/main/src/ragas/experiment.py): `src/ragas/experiment.py`
- [Evaluation optimizers](https://github.com/explodinggradients/ragas/tree/main/src/ragas/optimizers): `src/ragas/optimizers`

### Other components

- **LLM app developer**
- **LLM application**

## Connections

- LLM app developer → Evaluation workflow: starts evaluation (evidence: `README.md`)
- LLM application → Evaluation workflow: provides outputs (evidence: `README.md`)
- Evaluation workflow → Evaluation metrics: evaluates with (evidence: `README.md`)
- Evaluation metrics → Metric execution core: uses scoring core (evidence: `src/ragas/metrics/collections/base.py`)
- Metric execution core → LLM abstraction: uses LLM (evidence: `src/ragas/metrics/base.py`)
- Metric execution core → Embedding providers: uses embeddings (evidence: `src/ragas/metrics/base.py`)
- Metric execution core → Evaluation prompts: uses prompts (evidence: `src/ragas/metrics/base.py`)
- Metric execution core → Evaluation datasets: scores samples (evidence: `src/ragas/metrics/base.py`)
- Evaluation metrics → Metric results: returns scores (evidence: `src/ragas/metrics/collections/base.py`)
- LLM app developer → Test-set generation: requests test data (evidence: `README.md`)
- Test-set generation → Knowledge graph: uses graph (evidence: `src/ragas/testset/synthesizers/generate.py`)
- Test-set generation → Graph transforms: applies transforms (evidence: `src/ragas/testset/synthesizers/generate.py`)
- Test-set generation → Question synthesizers: generates samples (evidence: `src/ragas/testset/synthesizers/generate.py`)
- Test-set generation → LLM abstraction: uses LLM (evidence: `src/ragas/testset/synthesizers/generate.py`)
- Test-set generation → Embedding providers: uses embeddings (evidence: `src/ragas/testset/synthesizers/generate.py`)
- Test-set generation → Evaluation datasets: produces dataset (evidence: `src/ragas/testset/synthesizers/generate.py`)
- LLM abstraction → Model cache: caches requests (evidence: `src/ragas/llms/base.py`)
- Embedding providers → Model cache: caches embeddings (evidence: `src/ragas/embeddings/base.py`)
- LiteLLM provider → LLM abstraction: implements abstraction (evidence: `src/ragas/llms/litellm_llm.py`)
- Evaluation datasets → Dataset backends: exports and loads (evidence: `README.md`)
- Framework integrations → Production feedback: supports feedback loops (evidence: `README.md`)
- LLM app developer → Evaluation optimizers: improves evaluations (evidence: `README.md`)