explodinggradients/ragas

How explodinggradients/ragas works

Ragas evaluates LLM applications with built-in LLM-based and traditional metrics, and can generate test data when none exists. Its runtime also provides model adapters, dataset handling, framework/observability integrations, and optimization-related components. The principal workflow is documented in the README; generation and several internal dependencies are supported by indexed source references. Unsampled evaluator, backend, and integration internals limit relationship detail, so uncertain links are omitted.

Main entry points: generate.py Test-set generation · metrics/base.py Metric execution core · llms/base.py LLM abstraction · collections Evaluation metrics

Components 20

Evaluation

Evaluation workflow src/ragas/evaluation.py
Evaluation metrics src/ragas/metrics/collections
Metric execution core src/ragas/metrics/base.py
Metric results src/ragas/metrics/result.py
Evaluation prompts src/ragas/prompt

Test generation

Test-set generation src/ragas/testset/synthesizers/generate.py
Knowledge graph src/ragas/testset/graph.py
Graph transforms src/ragas/testset/transforms
Question synthesizers src/ragas/testset/synthesizers

Model components

LLM abstraction src/ragas/llms/base.py
LiteLLM provider src/ragas/llms/litellm_llm.py
Embedding providers src/ragas/embeddings/base.py

Data and results

Evaluation datasets src/ragas/dataset_schema.py
Dataset backends src/ragas/backends
Model cache src/ragas/cache.py

Integrations and improvement

Framework integrations src/ragas/integrations
Production feedback src/ragas/experiment.py
Evaluation optimizers src/ragas/optimizers

Other components

LLM app developer
LLM application

Connections 22

LLM app developer → Evaluation workflow: starts evaluation README.md
LLM application → Evaluation workflow: provides outputs README.md
Evaluation workflow → Evaluation metrics: evaluates with README.md
Evaluation metrics → Metric execution core: uses scoring core src/ragas/metrics/collections/base.py
Metric execution core → LLM abstraction: uses LLM src/ragas/metrics/base.py
Metric execution core → Embedding providers: uses embeddings src/ragas/metrics/base.py
Metric execution core → Evaluation prompts: uses prompts src/ragas/metrics/base.py
Metric execution core → Evaluation datasets: scores samples src/ragas/metrics/base.py
Evaluation metrics → Metric results: returns scores src/ragas/metrics/collections/base.py
LLM app developer → Test-set generation: requests test data README.md
Test-set generation → Knowledge graph: uses graph src/ragas/testset/synthesizers/generate.py
Test-set generation → Graph transforms: applies transforms src/ragas/testset/synthesizers/generate.py
Test-set generation → Question synthesizers: generates samples src/ragas/testset/synthesizers/generate.py
Test-set generation → LLM abstraction: uses LLM src/ragas/testset/synthesizers/generate.py
Test-set generation → Embedding providers: uses embeddings src/ragas/testset/synthesizers/generate.py
Test-set generation → Evaluation datasets: produces dataset src/ragas/testset/synthesizers/generate.py
LLM abstraction → Model cache: caches requests src/ragas/llms/base.py
Embedding providers → Model cache: caches embeddings src/ragas/embeddings/base.py
LiteLLM provider → LLM abstraction: implements abstraction src/ragas/llms/litellm_llm.py
Evaluation datasets → Dataset backends: exports and loads README.md
Framework integrations → Production feedback: supports feedback loops README.md
LLM app developer → Evaluation optimizers: improves evaluations README.md

More Python diagrams: mvt-project/mvt · laramies/theharvester · treeverse/dvc · coderamp-labs/gitingest · vibrantlabsai/ragas

Drawn by GitDiagram on September 30, 2026 from the repository's file tree, README and source files. Read as Markdown ·