# evidentlyai/evidently architecture

> Architecture diagram of the GitHub repository evidentlyai/evidently, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-04-05). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/evidentlyai/evidently
- Repository: https://github.com/evidentlyai/evidently

## Overview

- Core Library and API Layer – This includes the bulk of the business logic found in the “src/evidently” directory. It is subdivided into multiple modules:
- Calculation Engine (src/evidently/calculation_engine): Implements the metric implementations, supporting different numerical methods and statistical tests.
- Metrics, Test Suites, and Presets (src/evidently/metrics, src/evidently/test_preset, src/evidently/test_suite): Responsible for computing data drift, data quality, classification/regression metrics, etc.
- Pipeline & Runner (src/evidently/pipeline, src/evidently/runner): Manage the orchestration of evaluations, generating Reports or executing tests.
- Utilities and Configuration (src/evidently/_config.py, src/evidently/options, src/evidently/utils): Common helper functions, configuration management, and shared utilities.
- Command Line Interface (CLI) – Found in “src/evidently/cli”, it enables users to run evaluations from the command line.
- Collector and Integration Modules – Includes “src/evidently/collector” and supporting directories; these modules handle data ingestion, storage, and external integrations. Also, the numerous examples/integrations directories illustrate how Evidently connects with external services (Airflow, MLflow, Grafana, Postgres, etc.).
- External Deployment and Development Infrastructure – This includes Docker setup (various scripts and docker folders), GitHub workflows, container configurations, and CI/CD integrations that help package, test, and deploy the system.

##### Relationships and Data Flow
- The Core Library acts as the backend engine that gathers input data (both “reference” and “current”) and computes metrics and tests via the Calculation Engine. This output is then packaged into Reports or Test Suite results.
- The CLI or API interfaces (from the Core Library) are used to run evaluations through one-off scripts or automated pipelines.
- The Collector component ingests data (via REST API calls or file payloads) and forwards it to the Core Library modules.
- The Dashboard/UI layer retrieves or listens to evaluation results (using the API defined in “src/evidently/ui/api”) and visualizes them via interactive widgets and reports.
- The Integration Modules provide bridges between Evidently and external systems (such as Spark jobs, Airflow pipelines, Grafana dashboards, and MLflow experiments) shown in the “examples/integrations” segments. These indicate how Evidently can be plugged into existing ML infrastructures.
- CI/CD pipelines and configuration files (for Docker, GitHub actions, and pre-commit hooks) ensure that the system is continuously integrated and deployed.

##### Architectural Patterns and Design Principles
- Modularity and Plug-ability – Evidently is highly modular; components for metrics, test suites, and UI are separated cleanly. This encourages extensions (e.g., custom tests or new integrations).
- Layered Architecture – There is a clear separation of concerns between the core evaluation logic (calculations, metrics, tests), the delivery layer (CLI, API, UI), and external integrations.
- Use of Pipelines – The evaluation process is structured as a pipeline (from data input to processing to output generation), which is intrinsic to how reports and test suites are executed.
- Open Architecture for Integration – The design outlines simple APIs and registries (as seen in directories for registries and presets) to allow integration with third-party tools.

##### Relevant Technologies, Frameworks, and Libraries
- Python – The primary language, with standard libraries and popular frameworks for data handling (e.g., Pandas, scikit-learn).
- Pydantic – For configuration and data validation (note files like src/evidently/_pydantic_compat.py).
- Docker and CI/CD Tools – Evident from the docker configurations and GitHub workflows to support containerization and automated testing.
- Frontend Frameworks – In the UI directory, using TypeScript/React (visible from “tsx” files, themes, hooks, and router utilities within “ui/packages/evidently-ui-lib”) for the dashboard and interactive interfaces.

##### Additional Details
- Directory structure abstraction by maybe grouping components that are conceptually related (for instance, grouping all “src/evidently” modules together under a “Core Evaluation Engine” section, while drawing the “ui” folder as a separate “Frontend” block).

## Diagram (Mermaid)

```mermaid
flowchart TD
    %% Core Evaluation Engine
    subgraph "Core Evaluation Engine"
        calc["Calculation Engine\n(runs metrics and statistical tests)"]:::backend
        pipeline["Pipeline\n(manages evaluation pipelines)"]:::backend
        runner["Runner\n(orchestrates execution)"]:::backend
        utils["Utilities & Configuration\n(shared helpers & settings)"]:::backend
        core["Core Library & API\n(business logic & main API)"]:::backend
        metrics["Metrics\n(computes evaluation metrics)"]:::backend
        testsuite["Test Suites\n(executes system tests)"]:::backend
        presets["Presets\n(configured evaluation defaults)"]:::backend
        eval_out["Evaluation Outputs\n(Reports & Test Suite results)"]:::backend

        calc --> core
        pipeline --> core
        runner --> core
        utils --> core
        core --> metrics
        core --> testsuite
        core --> presets
        metrics --> eval_out
        testsuite --> eval_out
        presets --> eval_out
    end

    %% External Components
    cli["CLI\n(runs evaluations from command-line)"]:::cli
    collector["Collector & Integrations\n(ingests data & connects to external systems)"]:::integration
    ui["Dashboard / UI\n(visualizes evaluation reports)\n(backend & standalone frontend)"]:::frontend
    infra["Deployment & Dev Infra\n(Docker, CI/CD, Workflows)"]:::infra

    %% Relationships between external components and Core Evaluation Engine
    cli --> core
    collector --> core
    eval_out --> ui
    infra --> core
    infra --> cli
    infra --> collector
    infra --> ui

    %% Click Events for Core Evaluation Engine
    click core "https://github.com/evidentlyai/evidently/blob/main/src/evidently/__init__.py"
    click calc "https://github.com/evidentlyai/evidently/blob/main/src/evidently/calculation_engine/engine.py"
    click pipeline "https://github.com/evidentlyai/evidently/blob/main/src/evidently/pipeline/column_mapping.py"
    click runner "https://github.com/evidentlyai/evidently/blob/main/src/evidently/runner/runner.py"
    click utils "https://github.com/evidentlyai/evidently/blob/main/src/evidently/_config.py"
    click metrics "https://github.com/evidentlyai/evidently/tree/main/src/evidently/metrics"
    click testsuite "https://github.com/evidentlyai/evidently/blob/main/src/evidently/test_suite/test_suite.py"
    click presets "https://github.com/evidentlyai/evidently/blob/main/src/evidently/test_preset/test_preset.py"

    %% Click Events for External Components
    click cli "https://github.com/evidentlyai/evidently/blob/main/src/evidently/cli/main.py"
    click collector "https://github.com/evidentlyai/evidently/blob/main/src/evidently/collector/app.py"
    click ui "https://github.com/evidentlyai/evidently/tree/main/src/evidently/ui/api"
    click infra "https://github.com/evidentlyai/evidently/blob/main/docker/Dockerfile.service"

%% Styles
classDef backend fill:#AEDFF7,stroke:#000,stroke-width:1px;
classDef cli fill:#F7B7A3,stroke:#000,stroke-width:1px;
classDef integration fill:#C7F7C4,stroke:#000,stroke-width:1px;
classDef frontend fill:#FFD9B3,stroke:#000,stroke-width:1px;
classDef infra fill:#DAB3FF,stroke:#000,stroke-width:1px;
```
