# open-mmlab/mmdetection architecture

> Architecture diagram of the GitHub repository open-mmlab/mmdetection, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-11-09). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/open-mmlab/mmdetection
- Repository: https://github.com/open-mmlab/mmdetection

## Overview

MMDetection is a modular PyTorch computer-vision toolbox for detection and segmentation, with inference APIs and a broad catalog of model and dataset configurations. A user can supply an image and configured model to the inference API, which prepares inputs, runs the model, and returns predictions; the inferencer can also visualize results. The README documents object, instance, panoptic, and semi-supervised detection capabilities. Several model and training internals are represented at subsystem level because their implementations were not sampled; uncertain runtime links are omitted.

Main entry points: [`mmdet/models/detectors`](https://github.com/open-mmlab/mmdetection/tree/main/mmdet/models/detectors) (Detector Models), [`mmdet/datasets/transforms/transforms.py`](https://github.com/open-mmlab/mmdetection/blob/main/mmdet/datasets/transforms/transforms.py) (Data Transforms), [`mmdet/apis/inference.py`](https://github.com/open-mmlab/mmdetection/blob/main/mmdet/apis/inference.py) (Inference API), [`mmdet/apis/det_inferencer.py`](https://github.com/open-mmlab/mmdetection/blob/main/mmdet/apis/det_inferencer.py) (Detection Inferencer)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_interfaces["User Interfaces"]
  node_inference_api["Inference API<br/>[inference.py]"]
  node_inferencer["Detection Inferencer<br/>[det_inferencer.py]"]
  node_config_catalog["Model Configurations"]
end

subgraph group_models["Models and Components"]
  node_registry["Module Registry<br/>[registry.py]"]
  node_detectors["Detector Models"]
  node_queryinst["QueryInst<br/>[queryinst.py]"]
  node_proposal_head["Embedding RPN"]
  node_transformers["Transformer Layers"]
  node_positional_encoding["Positional Encoding"]
end

subgraph group_data["Data and Representations"]
  node_transforms["Data Transforms<br/>[transforms.py]"]
  node_bbox_types["Box Structures"]
  node_mask_types["Mask Structures<br/>[structures.py]"]
  node_data_samples["Detection Samples<br/>[det_data_sample.py]"]
end

subgraph group_capabilities["Tasks and Outputs"]
  node_evaluation["Evaluation"]
  node_visualizer["Local Visualizer"]
  node_pipeline_hook["Pipeline Switching"]
  node_xdecoder_caption["Image Captioning<br/>[image_caption.py]"]
end

node_user(("User"))

node_user -->|"requests inference"| node_inference_api
node_inference_api -->|"builds model"| node_registry
node_inference_api -->|"gets classes"| node_evaluation
node_inferencer -->|"renders predictions"| node_visualizer
node_queryinst -->|"inherits from"| node_detectors
node_proposal_head -->|"converts boxes"| node_bbox_types
node_transforms -->|"registers transforms"| node_registry
node_transforms -->|"uses box types"| node_bbox_types
node_transforms -->|"uses mask types"| node_mask_types
node_pipeline_hook -->|"rebuilds pipeline"| node_transforms
node_inferencer -.->|"returns predictions"| node_data_samples
node_detectors -.->|"produces samples"| node_data_samples
node_detectors -.->|"uses layers"| node_transformers
node_detectors -.->|"uses encoding"| node_positional_encoding
node_config_catalog -.->|"configures models"| node_detectors

click node_inference_api "https://github.com/open-mmlab/mmdetection/blob/main/mmdet/apis/inference.py"
click node_inferencer "https://github.com/open-mmlab/mmdetection/blob/main/mmdet/apis/det_inferencer.py"
click node_config_catalog "https://github.com/open-mmlab/mmdetection/tree/main/configs"
click node_registry "https://github.com/open-mmlab/mmdetection/blob/main/mmdet/registry.py"
click node_detectors "https://github.com/open-mmlab/mmdetection/tree/main/mmdet/models/detectors"
click node_queryinst "https://github.com/open-mmlab/mmdetection/blob/main/mmdet/models/detectors/queryinst.py"
click node_proposal_head "https://github.com/open-mmlab/mmdetection/blob/main/mmdet/models/dense_heads/embedding_rpn_head.py"
click node_transformers "https://github.com/open-mmlab/mmdetection/tree/main/mmdet/models/layers/transformer"
click node_positional_encoding "https://github.com/open-mmlab/mmdetection/blob/main/mmdet/models/layers/positional_encoding.py"
click node_transforms "https://github.com/open-mmlab/mmdetection/blob/main/mmdet/datasets/transforms/transforms.py"
click node_bbox_types "https://github.com/open-mmlab/mmdetection/tree/main/mmdet/structures/bbox"
click node_mask_types "https://github.com/open-mmlab/mmdetection/blob/main/mmdet/structures/mask/structures.py"
click node_data_samples "https://github.com/open-mmlab/mmdetection/blob/main/mmdet/structures/det_data_sample.py"
click node_evaluation "https://github.com/open-mmlab/mmdetection/tree/main/mmdet/evaluation"
click node_visualizer "https://github.com/open-mmlab/mmdetection/blob/main/mmdet/visualization/local_visualizer.py"
click node_pipeline_hook "https://github.com/open-mmlab/mmdetection/blob/main/mmdet/engine/hooks/pipeline_switch_hook.py"
click node_xdecoder_caption "https://github.com/open-mmlab/mmdetection/blob/main/projects/XDecoder/xdecoder/inference/image_caption.py"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_inference_api,node_inferencer,node_config_catalog,node_user toneBlue
class node_registry,node_detectors,node_queryinst,node_proposal_head,node_transformers,node_positional_encoding toneAmber
class node_transforms,node_bbox_types,node_mask_types,node_data_samples toneMint
class node_evaluation,node_visualizer,node_pipeline_hook,node_xdecoder_caption toneRose
```

## Components

### User Interfaces

- [Inference API](https://github.com/open-mmlab/mmdetection/blob/main/mmdet/apis/inference.py): `mmdet/apis/inference.py`
- [Detection Inferencer](https://github.com/open-mmlab/mmdetection/blob/main/mmdet/apis/det_inferencer.py): `mmdet/apis/det_inferencer.py`
- [Model Configurations](https://github.com/open-mmlab/mmdetection/tree/main/configs): `configs`

### Models and Components

- [Module Registry](https://github.com/open-mmlab/mmdetection/blob/main/mmdet/registry.py): `mmdet/registry.py`
- [Detector Models](https://github.com/open-mmlab/mmdetection/tree/main/mmdet/models/detectors): `mmdet/models/detectors`
- [QueryInst](https://github.com/open-mmlab/mmdetection/blob/main/mmdet/models/detectors/queryinst.py): `mmdet/models/detectors/queryinst.py`
- [Embedding RPN](https://github.com/open-mmlab/mmdetection/blob/main/mmdet/models/dense_heads/embedding_rpn_head.py): `mmdet/models/dense_heads/embedding_rpn_head.py`
- [Transformer Layers](https://github.com/open-mmlab/mmdetection/tree/main/mmdet/models/layers/transformer): `mmdet/models/layers/transformer`
- [Positional Encoding](https://github.com/open-mmlab/mmdetection/blob/main/mmdet/models/layers/positional_encoding.py): `mmdet/models/layers/positional_encoding.py`

### Data and Representations

- [Data Transforms](https://github.com/open-mmlab/mmdetection/blob/main/mmdet/datasets/transforms/transforms.py): `mmdet/datasets/transforms/transforms.py`
- [Box Structures](https://github.com/open-mmlab/mmdetection/tree/main/mmdet/structures/bbox): `mmdet/structures/bbox`
- [Mask Structures](https://github.com/open-mmlab/mmdetection/blob/main/mmdet/structures/mask/structures.py): `mmdet/structures/mask/structures.py`
- [Detection Samples](https://github.com/open-mmlab/mmdetection/blob/main/mmdet/structures/det_data_sample.py): `mmdet/structures/det_data_sample.py`

### Tasks and Outputs

- [Evaluation](https://github.com/open-mmlab/mmdetection/tree/main/mmdet/evaluation): `mmdet/evaluation`
- [Local Visualizer](https://github.com/open-mmlab/mmdetection/blob/main/mmdet/visualization/local_visualizer.py): `mmdet/visualization/local_visualizer.py`
- [Pipeline Switching](https://github.com/open-mmlab/mmdetection/blob/main/mmdet/engine/hooks/pipeline_switch_hook.py): `mmdet/engine/hooks/pipeline_switch_hook.py`
- [Image Captioning](https://github.com/open-mmlab/mmdetection/blob/main/projects/XDecoder/xdecoder/inference/image_caption.py): `projects/XDecoder/xdecoder/inference/image_caption.py`

### Other components

- **User**

## Connections

- User → Inference API: requests inference (evidence: `README.md`)
- Inference API → Module Registry: builds model (evidence: `mmdet/apis/inference.py`)
- Inference API → Evaluation: gets classes (evidence: `mmdet/apis/inference.py`)
- Detection Inferencer → Local Visualizer: renders predictions (evidence: `mmdet/apis/det_inferencer.py`)
- QueryInst → Detector Models: inherits from (evidence: `mmdet/models/detectors/queryinst.py`)
- Embedding RPN → Box Structures: converts boxes (evidence: `mmdet/models/dense_heads/embedding_rpn_head.py`)
- Data Transforms → Module Registry: registers transforms (evidence: `mmdet/datasets/transforms/transforms.py`)
- Data Transforms → Box Structures: uses box types (evidence: `mmdet/datasets/transforms/transforms.py`)
- Data Transforms → Mask Structures: uses mask types (evidence: `mmdet/datasets/transforms/transforms.py`)
- Pipeline Switching → Data Transforms: rebuilds pipeline (evidence: `mmdet/engine/hooks/pipeline_switch_hook.py`)
- Detection Inferencer → Detection Samples: returns predictions
- Detector Models → Detection Samples: produces samples
- Detector Models → Transformer Layers: uses layers
- Detector Models → Positional Encoding: uses encoding
- Model Configurations → Detector Models: configures models
