# mlcommons/dynabench architecture

> Architecture diagram of the GitHub repository mlcommons/dynabench, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-04-02). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/mlcommons/dynabench
- Repository: https://github.com/mlcommons/dynabench

## Overview

##### Main Components of the System
- Frontend Interfaces:
- There are two main frontend sections: one in “frontends/web” (the main user-facing web client) and documentation in “frontends/docs”. Emphasize that the web client interacts with the API and user services.
- Backend/API Servers:
- The “api” directory contains controllers, common utilities, cron tasks, migrations, and even specific modules for evaluation and building. This indicates that the backend not only serves API endpoints to support the frontends, but also runs asynchronous processes (e.g., via cron jobs).
- The “backend” directory also holds another layer of API endpoints (e.g., REST endpoints with middleware, domain services, and repositories) as well as separate modules for legacy “dynalab” applications supporting model building and evaluation.
- Evaluation and Building Modules:
- Within “api/evaluation” and “api/builder”, there are components for handling model evaluation, model deployment, and building tasks. They should be drawn as distinct processing services that also communicate with the persistent storage.
- Data Persistence and Migrations:
- The project contains a “migrations” folder (inside “api/migrations”) and model definitions (inside “api/models”). Indicate that a database (likely relational) is used, and that both the “api” and “backend” layers interact with it.
- External and Legacy Services:
- The “legacy” directories (annotators, mlcube, torchserve, etc.) represent older or supplementary components that may be integrated for handling specific machine learning models or tasks.
- Infrastructure and DevOps:
- There are docker-related files (Dockerfile, docker-compose.yaml, ECS task definitions) and configuration scripts (e.g., bootstrap.sh, various .env examples) that indicate containerization and deployment processes.

##### Relationships and Interactions
- Frontend Interaction:
- That the frontend may also serve static content (from “frontends” static assets) and documentation content.
- API Gateway / RESTful Services:
- The “api” and “backend” components both surface REST endpoints. The “api/controllers” handle various resources (auth, datasets, models, tasks, etc.), while “backend/app/api” has endpoints for builder and evaluation as well as other core services.
- Asynchronous and Task Processing:
- Model Building & Evaluation:
- The specialized modules in “api/builder”, “api/evaluation”, and the corresponding builder and evaluation services in “backend/app/domain/services/builder_and_evaluation” should be depicted as separate yet inter-related processing flows. These modules may also interact with external storage (e.g., S3) and messaging or email services (as implied by “api/infrastructure/email” and “backend/app/domain/helpers/email”).
- Data Persistence:

##### Architectural Patterns and Technologies
- Clear Separation of Concerns:
- The project separates concerns among frontend, API (controllers), domain logic (services), and data persistence (models/repositories).
- The existence of Dockerfiles, docker-compose setups, and ECS definitions speaks to containerized microservices or at least modular deployment.
- Modular and Layered Architecture:
- The system is modular—look at separate folders for controllers, domain logic, and infrastructure. This layered separation should be reflected by grouping related components.
- Asynchronous Processing:
- Cron jobs and background workers indicate an asynchronous processing pattern.
- Legacy Integration:
- The “legacy” folder components (annotators, mlcube, torchserve) may be integrated via adapter patterns or serve as optional processing pipelines.

##### Elements
- Clear Component Labels:
- From Frontend to API (user request flows).
- From API to Database (CRUD operations), email services, and external storage.
- Between asynchronous processors (cron jobs, worker tasks) and the main API services.
- Visual Distinctions:
- Additional Metadata:
- Brief labels indicating key technologies or frameworks (for example, Docker, Celery for asynchronous tasks, REST for API communications).

## Diagram (Mermaid)

```mermaid
flowchart TD
    subgraph "Frontend Interfaces"
        FW["Frontend Web Client"]:::frontend
        DF["Documentation Frontend"]:::frontend
    end

    subgraph "Backend & API"
        API_Controller["API Controllers & Common Utilities"]:::backend
        Backend_API["Backend API Endpoints & Domain Services"]:::backend
        EvalModules["Evaluation Modules"]:::processing
        BuildModules["Building Modules"]:::processing
        CronJob["Cron Job / Asynchronous Processing"]:::processing
        EmailService["Email & External Infrastructure Services"]:::processing
    end

    subgraph "Data Persistence"
        DataMigr["Data Migrations & Persistence Models"]:::database
        ReposDB["Repositories & Database Interaction"]:::database
    end

    subgraph "Legacy Services"
        LegacyServices["Legacy Services (Annotators, Mlcube, Torchserve)"]:::legacy
    end

    subgraph "Infrastructure & DevOps"
        DevOps["Infrastructure & DevOps (Containerization & Deployment)"]:::devops
    end

    %% Connections between Frontend and API
    FW -->|"request"| API_Controller
    DF -->|"readsDocs"| API_Controller

    %% API Layer connections
    API_Controller -->|"forwards"| Backend_API
    API_Controller -->|"triggersEval"| EvalModules
    API_Controller -->|"triggersBuild"| BuildModules

    Backend_API -->|"processesEvalAndBuild"| EvalModules
    Backend_API -->|"processesEvalAndBuild"| BuildModules
    Backend_API -->|"readsWrites"| ReposDB

    %% Data Persistence interactions
    EvalModules -->|"DBQuery"| DataMigr
    BuildModules -->|"DBQuery"| DataMigr

    %% Asynchronous Processing
    CronJob -->|"executesTasks"| Backend_API

    %% Email Notifications
    EmailService -->|"notifies"| Backend_API
    EmailService -->|"alertsEval"| EvalModules

    %% Legacy integrations
    LegacyServices -->|"integrates"| Backend_API

    %% DevOps deployments
    DevOps -->|"deploys"| FW
    DevOps -->|"deploys"| DF
    DevOps -->|"deploys"| API_Controller
    DevOps -->|"deploys"| Backend_API
    DevOps -->|"deploys"| DataMigr
    DevOps -->|"deploys"| CronJob

    %% Click Events
    click FW "https://github.com/mlcommons/dynabench/tree/main/frontends/web"
    click DF "https://github.com/mlcommons/dynabench/tree/main/frontends/docs"
    click API_Controller "https://github.com/mlcommons/dynabench/tree/main/api/controllers"
    click API_Controller "https://github.com/mlcommons/dynabench/tree/main/api/common"
    click CronJob "https://github.com/mlcommons/dynabench/tree/main/api/cron"
    click EvalModules "https://github.com/mlcommons/dynabench/tree/main/api/evaluation"
    click BuildModules "https://github.com/mlcommons/dynabench/tree/main/api/builder"
    click DataMigr "https://github.com/mlcommons/dynabench/tree/main/api/migrations"
    click DataMigr "https://github.com/mlcommons/dynabench/tree/main/api/models"
    click Backend_API "https://github.com/mlcommons/dynabench/tree/main/backend/app/api"
    click Backend_API "https://github.com/mlcommons/dynabench/tree/main/backend/app/domain/services/builder_and_evaluation"
    click ReposDB "https://github.com/mlcommons/dynabench/tree/main/backend/app/infrastructure/repositories"
    click EmailService "https://github.com/mlcommons/dynabench/tree/main/api/infrastructure/email"
    click EmailService "https://github.com/mlcommons/dynabench/tree/main/backend/app/domain/helpers/email"
    click LegacyServices "https://github.com/mlcommons/dynabench/tree/main/legacy/annotators"
    click LegacyServices "https://github.com/mlcommons/dynabench/tree/main/legacy/mlcube"
    click LegacyServices "https://github.com/mlcommons/dynabench/tree/main/legacy/torchserve"
    click DevOps "https://github.com/mlcommons/dynabench/tree/main/Dockerfile"
    click DevOps "https://github.com/mlcommons/dynabench/blob/main/docker-compose.yaml"
    click DevOps "https://github.com/mlcommons/dynabench/tree/main/api/devops"
    click DevOps "https://github.com/mlcommons/dynabench/tree/main/backend/dockerfiles"
    click DevOps "https://github.com/mlcommons/dynabench/blob/main/backend/docker-compose-queue.yaml"

    %% Styles
    classDef frontend fill:#B3E5FC,stroke:#0288D1,stroke-width:2px;
    classDef backend fill:#FFECB3,stroke:#F57C00,stroke-width:2px;
    classDef processing fill:#FFF9C4,stroke:#FBC02D,stroke-width:2px;
    classDef database fill:#C8E6C9,stroke:#388E3C,stroke-width:2px;
    classDef legacy fill:#E0E0E0,stroke:#757575,stroke-width:2px;
    classDef devops fill:#D1C4E9,stroke:#7B1FA2,stroke-width:2px;
```
