# open-mmlab/mmpretrain architecture

> Architecture diagram of the GitHub repository open-mmlab/mmpretrain, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-05-23). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/open-mmlab/mmpretrain
- Repository: https://github.com/open-mmlab/mmpretrain

## Overview

##### Project Type and Purpose
- This is a full‐fledged Python toolbox (library/framework) for deep learning pre‐training and multi‐modal tasks, built on PyTorch. It is not a standalone service but a pluggable SDK providing APIs, command‐line scripts, configs, and tools for dataset preparation, training, inference, and evaluation.

##### High‐Level Directory Analysis
- Top‐level:
- mmpretrain/           → core Python package (APIs, configs, datasets, engine, evaluation, models, utilities, visualization)
- configs/              → model/dataset/training configuration files (YAML/Python) for many backbones and algorithms
- tools/                → CLI scripts (train.py, test.py), model converters, dataset converters, visualization tools
- demo/, projects/      → example usage (image_demo.py, gradio demos, custom projects)
- docs/                 → Sphinx documentation source (en/zh_CN)
- docker/, .circleci/, .github/ → CI/CD and container orchestration
- tests/                → unit/integration tests covering all modules

##### Core Architectural Patterns
- Plugin/Registry Pattern: modules (datasets, models, hooks, metrics) register themselves at import via a central `registry.py`.
- Config‐Driven Pipeline: user‐provided Python/YAML configs instantiate data pipelines, models, optimizers, runners.
- Engine‐Runner Pattern: training/inference is orchestrated by engine runners (hooks, schedulers, runners).

##### Main Components
a. User Interface Layer
- CLI Scripts (tools/train.py, tools/test.py, demo/image_demo.py)
- Programmatic APIs (mmpretrain/apis/*.py)
- Configuration Files (configs/*)
b. Core Framework
- Config Parser & Builder (mmpretrain/registry.py, mmpretrain/apis/utils.py)
- Dataset Management (mmpretrain/datasets/*)
- Model Zoo & Builders
- Backbones (mmpretrain/models/backbones/*)
- Heads, Necks, Retrievers, Losses, etc.
- Training Engine (mmpretrain/engine/*)
- Runners, Hooks, Optimizers, Schedulers
- Evaluation Engine (mmpretrain/evaluation/*)
- Metric computations, visual grounding, retrieval metrics
- Utilities & Visualization (mmpretrain/utils/*, mmpretrain/visualization/*)
c. External Dependencies
- PyTorch, torchvision, CUDA, mmcv, mmengine
- Docker container for serving (docker/serve)
- CI/CD (CircleCI, GitHub Actions)
d. Outputs & Artifacts
- Checkpoints, logs, TensorBoard, model zoo entries
- Evaluation reports, visualizations

##### Essential Elements
- Labels on each component (exact package or directory name)
- Registry as central connector for plugin modules
- Config files → Core pipeline instantiation
- Data → Model → Engine → Evaluation
- Optional: Docker serving & CI/CD pipelines on the side

## Diagram (Mermaid)

```mermaid
flowchart TD
    %% User Interface Layer
    subgraph "User Interface Layer"
        direction TB
        TrainScript["tools/train.py"]:::ui
        TestScript["tools/test.py"]:::ui
        DemoScript["demo/image_demo.py"]:::ui
        APIs["mmpretrain/apis/"]:::ui
        ConfigFiles["configs/"]:::ui
    end

    %% Core Framework
    subgraph "Core Framework"
        direction TB
        subgraph "Config & Registry"
            direction TB
            ConfigParser["registry.py"]:::core
            UtilsBuilder["apis/utils.py"]:::core
        end
        subgraph "Plugin Registry"
            direction TB
            RegistryHub["Registry"]:::plugin
        end
        subgraph "Dataset Management"
            direction TB
            Datasets["mmpretrain/datasets/"]:::plugin
        end
        subgraph "Model Zoo & Builders"
            direction TB
            ModelBuilder["builder.py"]:::core
            Backbones["backbones/"]:::plugin
            Heads["heads/"]:::plugin
            Necks["necks/"]:::plugin
            Retrievers["retrievers/"]:::plugin
            Losses["losses/"]:::plugin
        end
        subgraph "Training Engine"
            direction TB
            Runners["runners/"]:::core
            Hooks["hooks/"]:::plugin
            Optimizers["optimizers/"]:::plugin
            Schedulers["schedulers/"]:::plugin
        end
        subgraph "Evaluation Engine"
            direction TB
            Metrics["metrics/"]:::plugin
            Functional["functional/"]:::plugin
        end
        subgraph "Utilities & Visualization"
            direction TB
            Utils["mmpretrain/utils/"]:::core
            Visualization["mmpretrain/visualization/"]:::core
        end
    end

    %% External Dependencies
    subgraph "External Dependencies"
        direction TB
        PyTorch["PyTorch, CUDA, torchvision"]:::external
        MMCV["mmcv, mmengine"]:::external
        DockerServe["docker/serve/"]:::external
        CICD[".github/workflows/, .circleci/"]:::external
        Docs["docs/"]:::external
        Tests["tests/"]:::external
        Projects["projects/"]:::external
    end

    %% Artifacts
    subgraph "Outputs & Artifacts"
        direction TB
        Checkpoints["Checkpoints"]:::artifact
        Logs["Logs & TensorBoard"]:::artifact
        Reports["Evaluation Reports & Visualizations"]:::artifact
    end

    %% Connections
    TrainScript -->|"load config"| ConfigFiles
    TestScript -->|"load config"| ConfigFiles
    DemoScript -->|"call APIs"| APIs
    APIs -->|"parse config"| ConfigParser
    ConfigFiles -->|"feed to"| ConfigParser

    ConfigParser -->|"instantiate modules"| RegistryHub
    RegistryHub --> Datasets
    RegistryHub --> Backbones
    RegistryHub --> Heads
    RegistryHub --> Necks
    RegistryHub --> Retrievers
    RegistryHub --> Losses

    Datasets -->|"build data loader"| Runners
    Backbones --> ModelBuilder
    Heads --> ModelBuilder
    Necks --> ModelBuilder
    Retrievers --> ModelBuilder
    Losses --> ModelBuilder

    ModelBuilder --> Runners
    Runners -->|"forward pass"| ModelBuilder
    Runners --> Optimizers
    Runners --> Schedulers
    Runners --> Hooks
    Hooks --> Utils
    Runners -->|"call evaluation"| Metrics
    Runners --> Functional

    Metrics --> Reports
    Functional --> Reports
    Runners --> Checkpoints
    Hooks --> Logs

    PyTorch --> Runners
    MMCV --> Utils
    DockerServe -.-> Reports
    DockerServe -->|"serving model"| Checkpoints
    CICD -.-> Tests
    CICD -.-> Docs

    Projects --> DemoScript
    Projects -.-> APIs

    %% Click Events
    click TrainScript "https://github.com/open-mmlab/mmpretrain/blob/main/tools/train.py"
    click TestScript "https://github.com/open-mmlab/mmpretrain/blob/main/tools/test.py"
    click DemoScript "https://github.com/open-mmlab/mmpretrain/blob/main/demo/image_demo.py"
    click APIs "https://github.com/open-mmlab/mmpretrain/tree/main/mmpretrain/apis/"
    click ConfigFiles "https://github.com/open-mmlab/mmpretrain/tree/main/configs/"
    click ConfigParser "https://github.com/open-mmlab/mmpretrain/blob/main/mmpretrain/registry.py"
    click UtilsBuilder "https://github.com/open-mmlab/mmpretrain/blob/main/mmpretrain/apis/utils.py"
    click Datasets "https://github.com/open-mmlab/mmpretrain/tree/main/mmpretrain/datasets/"
    click ModelBuilder "https://github.com/open-mmlab/mmpretrain/blob/main/mmpretrain/models/builder.py"
    click Backbones "https://github.com/open-mmlab/mmpretrain/tree/main/mmpretrain/models/backbones/"
    click Heads "https://github.com/open-mmlab/mmpretrain/tree/main/mmpretrain/models/heads/"
    click Necks "https://github.com/open-mmlab/mmpretrain/tree/main/mmpretrain/models/necks/"
    click Retrievers "https://github.com/open-mmlab/mmpretrain/tree/main/mmpretrain/models/retrievers/"
    click Losses "https://github.com/open-mmlab/mmpretrain/tree/main/mmpretrain/models/losses/"
    click Runners "https://github.com/open-mmlab/mmpretrain/tree/main/mmpretrain/engine/runners/"
    click Hooks "https://github.com/open-mmlab/mmpretrain/tree/main/mmpretrain/engine/hooks/"
    click Optimizers "https://github.com/open-mmlab/mmpretrain/tree/main/mmpretrain/engine/optimizers/"
    click Schedulers "https://github.com/open-mmlab/mmpretrain/tree/main/mmpretrain/engine/schedulers/"
    click Metrics "https://github.com/open-mmlab/mmpretrain/tree/main/mmpretrain/evaluation/metrics/"
    click Functional "https://github.com/open-mmlab/mmpretrain/tree/main/mmpretrain/evaluation/functional/"
    click Utils "https://github.com/open-mmlab/mmpretrain/tree/main/mmpretrain/utils/"
    click Visualization "https://github.com/open-mmlab/mmpretrain/tree/main/mmpretrain/visualization/"
    click DockerServe "https://github.com/open-mmlab/mmpretrain/tree/main/docker/serve/"
    click CICD "https://github.com/open-mmlab/mmpretrain/tree/main/.github/workflows/"
    click Projects "https://github.com/open-mmlab/mmpretrain/tree/main/projects/"

    %% Styles
    classDef ui fill:#a2f5a2,stroke:#333,stroke-width:1px
    classDef core fill:#a2d0f5,stroke:#333,stroke-width:1px
    classDef plugin fill:#f5e68a,stroke:#333,stroke-width:1px
    classDef external fill:#cccccc,stroke:#333,stroke-width:1px
    classDef artifact fill:#f5b7b1,stroke:#333,stroke-width:1px
```
