# open-mmlab/mmaction architecture

> Architecture diagram of the GitHub repository open-mmlab/mmaction, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-10-04). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/open-mmlab/mmaction
- Repository: https://github.com/open-mmlab/mmaction

## Overview

MMAction is a PyTorch toolbox for action understanding. Its user workflow is to prepare video data, select a configured recognition or detection model, train or run inference, and evaluate predictions. The product spans trimmed-video recognition, temporal localization, and spatiotemporal detection, with shared datasets, model components, and bounding-box utilities. The graph preserves those documented capabilities; several subsystem links are documented only at a high level because their implementation files were not sampled, so those links are marked unverified.

Main entry points: [`mmaction/apis/train.py`](https://github.com/open-mmlab/mmaction/blob/master/mmaction/apis/train.py) (Training API), [`mmaction/datasets`](https://github.com/open-mmlab/mmaction/tree/master/mmaction/datasets) (Video datasets), [`tools/eval_localize_results.py`](https://github.com/open-mmlab/mmaction/blob/master/tools/eval_localize_results.py) (Localization evaluator), [`mmaction/models/recognizers/TSN3D.py`](https://github.com/open-mmlab/mmaction/blob/master/mmaction/models/recognizers/TSN3D.py) (TSN3D)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_data["Video data"]
  node_prep["Data preparation"]
  node_dataset_families["Video datasets"]
  node_ssn_dataset["SSN proposals<br/>[ssn_dataset.py]"]
  node_transforms["Video transforms<br/>[transforms.py]"]
  node_loader["Batch loading<br/>[build_loader.py]"]
end

subgraph group_models["Model components"]
  node_model_registry["Model registry<br/>[registry.py]"]
  node_model_builder["Model builder<br/>[builder.py]"]
  node_recognizers["Action recognizers"]
  node_tsn3d["TSN3D<br/>[TSN3D.py]"]
  node_backbones["Video backbones"]
  node_bninception["BNInception<br/>[bninception.py]"]
  node_detectors["Spatial detectors"]
  node_localizers["Temporal localizers"]
  node_losses["Training losses"]
end

subgraph group_runtime["Training and inference"]
  node_train_tools["Training entrypoints"]
  node_train_api["Training API<br/>[train.py]"]
end

subgraph group_analysis["Detection and evaluation"]
  node_bbox["Box targets"]
  node_anchor_targets["Anchor targets<br/>[anchor_target.py]"]
  node_eval_hooks["Validation hooks<br/>[eval_hooks.py]"]
  node_localize_eval["Localization metrics<br/>[localize_utils.py]"]
  node_eval_tool["Localization evaluator"]
end

node_user(("Researcher"))
node_config["Model configuration"]

node_user -->|"prepares data"| node_prep
node_prep -.->|"feeds datasets"| node_dataset_families
node_user -.->|"starts training"| node_train_tools
node_config -.->|"configures run"| node_train_api
node_train_tools -.->|"invokes"| node_train_api
node_train_api -->|"builds batches"| node_loader
node_train_api -->|"trains on"| node_dataset_families
node_train_api -->|"forwards batches"| node_recognizers
node_model_builder -->|"looks up modules"| node_model_registry
node_tsn3d -->|"builds backbone"| node_model_builder
node_tsn3d -->|"extracts features"| node_backbones
node_bninception -->|"registers backbone"| node_model_registry
node_recognizers -.->|"predicts actions"| node_user
node_detectors -.->|"uses box utilities"| node_bbox
node_anchor_targets -->|"assigns box targets"| node_bbox
node_localizers -.->|"localizes proposals"| node_ssn_dataset
node_ssn_dataset -->|"uses transforms"| node_transforms
node_train_api -.->|"optimizes outputs"| node_losses
node_train_api -->|"registers validation"| node_eval_hooks
node_eval_tool -->|"builds test dataset"| node_dataset_families
node_eval_tool -->|"evaluates detections"| node_localize_eval
node_eval_hooks -->|"reads annotations"| node_dataset_families
node_eval_tool -.->|"reports metrics"| node_user

click node_prep "https://github.com/open-mmlab/mmaction/tree/master/data_tools"
click node_dataset_families "https://github.com/open-mmlab/mmaction/tree/master/mmaction/datasets"
click node_ssn_dataset "https://github.com/open-mmlab/mmaction/blob/master/mmaction/datasets/ssn_dataset.py"
click node_transforms "https://github.com/open-mmlab/mmaction/blob/master/mmaction/datasets/transforms.py"
click node_loader "https://github.com/open-mmlab/mmaction/blob/master/mmaction/datasets/loader/build_loader.py"
click node_model_registry "https://github.com/open-mmlab/mmaction/blob/master/mmaction/models/registry.py"
click node_model_builder "https://github.com/open-mmlab/mmaction/blob/master/mmaction/models/builder.py"
click node_recognizers "https://github.com/open-mmlab/mmaction/tree/master/mmaction/models/recognizers"
click node_tsn3d "https://github.com/open-mmlab/mmaction/blob/master/mmaction/models/recognizers/TSN3D.py"
click node_backbones "https://github.com/open-mmlab/mmaction/tree/master/mmaction/models/tenons/backbones"
click node_bninception "https://github.com/open-mmlab/mmaction/blob/master/mmaction/models/tenons/backbones/bninception.py"
click node_detectors "https://github.com/open-mmlab/mmaction/tree/master/mmaction/models/detectors"
click node_localizers "https://github.com/open-mmlab/mmaction/tree/master/mmaction/models/localizers"
click node_losses "https://github.com/open-mmlab/mmaction/tree/master/mmaction/losses"
click node_train_tools "https://github.com/open-mmlab/mmaction/blob/master/tools/train_recognizer.py"
click node_train_api "https://github.com/open-mmlab/mmaction/blob/master/mmaction/apis/train.py"
click node_bbox "https://github.com/open-mmlab/mmaction/tree/master/mmaction/core/bbox2d"
click node_anchor_targets "https://github.com/open-mmlab/mmaction/blob/master/mmaction/core/anchor2d/anchor_target.py"
click node_eval_hooks "https://github.com/open-mmlab/mmaction/blob/master/mmaction/core/evaluation/eval_hooks.py"
click node_localize_eval "https://github.com/open-mmlab/mmaction/blob/master/mmaction/core/evaluation/localize_utils.py"
click node_eval_tool "https://github.com/open-mmlab/mmaction/blob/master/tools/eval_localize_results.py"
click node_config "https://github.com/open-mmlab/mmaction/tree/master/configs"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_prep,node_dataset_families,node_ssn_dataset,node_transforms,node_loader toneBlue
class node_model_registry,node_model_builder,node_recognizers,node_tsn3d,node_backbones,node_bninception,node_detectors,node_localizers,node_losses toneAmber
class node_train_tools,node_train_api toneMint
class node_bbox,node_anchor_targets,node_eval_hooks,node_localize_eval,node_eval_tool toneRose
class node_user,node_config toneIndigo
```

## Components

### Video data

- [Data preparation](https://github.com/open-mmlab/mmaction/tree/master/data_tools): `data_tools`
- [Video datasets](https://github.com/open-mmlab/mmaction/tree/master/mmaction/datasets): `mmaction/datasets`
- [SSN proposals](https://github.com/open-mmlab/mmaction/blob/master/mmaction/datasets/ssn_dataset.py): `mmaction/datasets/ssn_dataset.py`
- [Video transforms](https://github.com/open-mmlab/mmaction/blob/master/mmaction/datasets/transforms.py): `mmaction/datasets/transforms.py`
- [Batch loading](https://github.com/open-mmlab/mmaction/blob/master/mmaction/datasets/loader/build_loader.py): `mmaction/datasets/loader/build_loader.py`

### Model components

- [Model registry](https://github.com/open-mmlab/mmaction/blob/master/mmaction/models/registry.py): `mmaction/models/registry.py`
- [Model builder](https://github.com/open-mmlab/mmaction/blob/master/mmaction/models/builder.py): `mmaction/models/builder.py`
- [Action recognizers](https://github.com/open-mmlab/mmaction/tree/master/mmaction/models/recognizers): `mmaction/models/recognizers`
- [TSN3D](https://github.com/open-mmlab/mmaction/blob/master/mmaction/models/recognizers/TSN3D.py): `mmaction/models/recognizers/TSN3D.py`
- [Video backbones](https://github.com/open-mmlab/mmaction/tree/master/mmaction/models/tenons/backbones): `mmaction/models/tenons/backbones`
- [BNInception](https://github.com/open-mmlab/mmaction/blob/master/mmaction/models/tenons/backbones/bninception.py): `mmaction/models/tenons/backbones/bninception.py`
- [Spatial detectors](https://github.com/open-mmlab/mmaction/tree/master/mmaction/models/detectors): `mmaction/models/detectors`
- [Temporal localizers](https://github.com/open-mmlab/mmaction/tree/master/mmaction/models/localizers): `mmaction/models/localizers`
- [Training losses](https://github.com/open-mmlab/mmaction/tree/master/mmaction/losses): `mmaction/losses`

### Training and inference

- [Training entrypoints](https://github.com/open-mmlab/mmaction/blob/master/tools/train_recognizer.py): `tools/train_recognizer.py`
- [Training API](https://github.com/open-mmlab/mmaction/blob/master/mmaction/apis/train.py): `mmaction/apis/train.py`

### Detection and evaluation

- [Box targets](https://github.com/open-mmlab/mmaction/tree/master/mmaction/core/bbox2d): `mmaction/core/bbox2d`
- [Anchor targets](https://github.com/open-mmlab/mmaction/blob/master/mmaction/core/anchor2d/anchor_target.py): `mmaction/core/anchor2d/anchor_target.py`
- [Validation hooks](https://github.com/open-mmlab/mmaction/blob/master/mmaction/core/evaluation/eval_hooks.py): `mmaction/core/evaluation/eval_hooks.py`
- [Localization metrics](https://github.com/open-mmlab/mmaction/blob/master/mmaction/core/evaluation/localize_utils.py): `mmaction/core/evaluation/localize_utils.py`
- [Localization evaluator](https://github.com/open-mmlab/mmaction/blob/master/tools/eval_localize_results.py): `tools/eval_localize_results.py`

### Other components

- **Researcher**
- [Model configuration](https://github.com/open-mmlab/mmaction/tree/master/configs): `configs`

## Connections

- Researcher → Data preparation: prepares data
- Data preparation → Video datasets: feeds datasets
- Researcher → Training entrypoints: starts training
- Model configuration → Training API: configures run
- Training entrypoints → Training API: invokes
- Training API → Batch loading: builds batches (evidence: `mmaction/apis/train.py`)
- Training API → Video datasets: trains on (evidence: `mmaction/apis/train.py`)
- Training API → Action recognizers: forwards batches (evidence: `mmaction/apis/train.py`)
- Model builder → Model registry: looks up modules
- TSN3D → Model builder: builds backbone (evidence: `mmaction/models/recognizers/TSN3D.py`)
- TSN3D → Video backbones: extracts features (evidence: `mmaction/models/recognizers/TSN3D.py`)
- BNInception → Model registry: registers backbone (evidence: `mmaction/models/tenons/backbones/bninception.py`)
- Action recognizers → Researcher: predicts actions
- Spatial detectors → Box targets: uses box utilities
- Anchor targets → Box targets: assigns box targets (evidence: `mmaction/core/anchor2d/anchor_target.py`)
- Temporal localizers → SSN proposals: localizes proposals
- SSN proposals → Video transforms: uses transforms (evidence: `mmaction/datasets/ssn_dataset.py`)
- Training API → Training losses: optimizes outputs
- Training API → Validation hooks: registers validation (evidence: `mmaction/apis/train.py`)
- Localization evaluator → Video datasets: builds test dataset (evidence: `tools/eval_localize_results.py`)
- Localization evaluator → Localization metrics: evaluates detections (evidence: `tools/eval_localize_results.py`)
- Validation hooks → Video datasets: reads annotations (evidence: `mmaction/core/evaluation/eval_hooks.py`)
- Localization evaluator → Researcher: reports metrics
