How open-mmlab/mmengine works
Purpose and entry points
MMEngine is a Python/PyTorch foundation library for configurable deep-learning training, validation, testing, inference, and experiment operations. Consumers import its public package (mmengine/__init__.py) and typically construct Runner programmatically or from configuration; examples include examples/segmentation/train.py and examples/llama2/fsdp_finetune.py. It is a reusable library, not a standalone service or web application.
Configuration and dependency construction
The configuration layer loads/merges Python, JSON, and YAML configurations, including lazy Python configuration support. The registry and default-scope system turn typed config dictionaries into project or third-party objects, decoupling engine orchestration from model, dataset, metric, hook, and optimizer implementations.
• mmengine/config/config.py
• mmengine/config/lazy.py
• mmengine/registry/registry.py
Training orchestration
Runner is the central control-plane object: it owns runtime state and coordinates model execution, dataloaders, loops, evaluators, optimization, hooks, checkpointing, and logging. Standard and flexible runner implementations delegate iteration/epoch behavior to loop classes. Primary flow is: configuration/objects → runner → train/validation/test loops → model and optimizer steps → hooks, evaluators, and output backends.
• mmengine/runner/runner.py
• mmengine/runner/_flexible_runner.py
• mmengine/runner/loops.py
Model, data, and evaluation boundary
Applications supply PyTorch-compatible models through BaseModel, which standardizes forward modes such as loss and prediction plus data preprocessing. Dataset abstractions, wrappers, and samplers provide data to PyTorch dataloaders. Evaluation converts per-batch predictions into metric state/results; application-specific models and metrics remain outside the engine.
• mmengine/model/base_model/base_model.py
• mmengine/dataset/base_dataset.py
• mmengine/evaluator/evaluator.py
Optimization and extensibility lifecycle
Optimizer wrappers encapsulate backward propagation, gradient accumulation/clipping, AMP, and parameter updates; schedulers change optimization parameters over time. Hooks are ordered lifecycle extensions around runner and loop events, covering checkpointing, scheduling, timers, EMA, profiling, and logging without embedding these concerns in core loops.
• mmengine/optim/optimizer/optimizer_wrapper.py
• mmengine/optim/scheduler/param_scheduler.py
• mmengine/hooks/hook.py
Scale-out strategies and distributed execution
PyTorch distributed utilities and model wrappers support multi-process/device execution. The strategy subsystem abstracts single-device, standard distributed, FSDP, DeepSpeed, and ColossalAI setup, allowing the runner to operate across differing large-model runtimes. Activation checkpointing and AMP live near runner execution.
• mmengine/_strategy/base.py
• mmengine/_strategy/deepspeed.py
• mmengine/_strategy/fsdp.py
Persistence, observability, and external integrations
Checkpoint facilities persist and resume training state. Logging aggregates scalar/runtime state through a message hub; visualization routes artifacts and metrics through pluggable backends. File I/O abstracts local files, HTTP, LMDB, Memcached, and Petrel storage. Supported integrations include TensorBoard, Weights & Biases, MLflow, ClearML, Neptune, DVCLive, and Aim through visualization/logging backends; PyTorch is the primary runtime, with optional DeepSpeed, FSDP, and ColossalAI.
• mmengine/runner/checkpoint.py
• mmengine/visualization/vis_backend.py
• mmengine/fileio/io.py
Main entry points: runner.py Runner · loops.py Train / Val / Test Loops · hook.py Hooks · config.py Configuration
Components 19
Configuration & Construction
- Configuration · Loads and merges Python, JSON, YAML, and lazy configs.
mmengine/config/config.py - Lazy Config
mmengine/config/lazy.py - Registry & Scope · Builds typed configuration objects and resolves project scopes.
mmengine/registry/registry.py
Runner Execution
- Runner · Owns runtime state and coordinates execution components.
mmengine/runner/runner.py - Flexible Runner
mmengine/runner/_flexible_runner.py - Train / Val / Test Loops · Implements iteration and epoch behavior.
mmengine/runner/loops.py - BaseModel · Standardizes preprocessing, loss, and prediction forward modes.
mmengine/model/base_model/base_model.py - BaseDataset · Supplies dataset abstractions for PyTorch dataloaders.
mmengine/dataset/base_dataset.py - Evaluator · Accumulates predictions into metric results.
mmengine/evaluator/evaluator.py - Execution Strategies · Abstracts single-device, distributed, FSDP, DeepSpeed, and ColossalAI setup.
mmengine/_strategy/base.py - DeepSpeed Strategy
mmengine/_strategy/deepspeed.py - FSDP Strategy
mmengine/_strategy/fsdp.py
Optimization & Extensions
- Optimizer Wrapper · Manages backward pass, AMP, gradients, and parameter updates.
mmengine/optim/optimizer/optimizer_wrapper.py - Parameter Scheduler
mmengine/optim/scheduler/param_scheduler.py - Hooks · Ordered callbacks for runner and loop events.
mmengine/hooks/hook.py
Persistence & Integrations
- Checkpointing · Persists and resumes training state.
mmengine/runner/checkpoint.py - Visualization Backends · Routes metrics and artifacts to pluggable integrations.
mmengine/visualization/vis_backend.py - File I/O · Accesses local and remote storage backends.
mmengine/fileio/io.py
Other components
- Applications · Programs construct and run MMEngine training workflows.
Connections 19
- Applications → Configuration: loads
- Applications → Runner: constructs
- Configuration → Lazy Config: supports
- Configuration → Registry & Scope: provides specs to
- Registry & Scope → Runner: builds components for
- Runner → Flexible Runner: offers variant
- Runner → Train / Val / Test Loops: delegates execution to
- Runner → Execution Strategies: selects runtime
- Execution Strategies → DeepSpeed Strategy: implements
- Execution Strategies → FSDP Strategy: implements
- Train / Val / Test Loops → BaseDataset: reads batches from
- Train / Val / Test Loops → BaseModel: executes
- Train / Val / Test Loops → Optimizer Wrapper: updates through
- Train / Val / Test Loops → Evaluator: submits predictions to
- Runner → Hooks: emits lifecycle events to
- Hooks → Parameter Scheduler: drives
- Hooks → Checkpointing: triggers
- Hooks → Visualization Backends: reports through
- Checkpointing → File I/O: stores via