How mayank-atheer/detectron2 works
Project Type and Purpose
• Detectron2 is a modular, research-grade computer vision library built on PyTorch.
• It acts as both a toolkit for building object detection/segmentation models and a framework for training, evaluation, and model export.
Top-Level Architecture Components
a. Configuration System
• YAML/py configs under configs/ and projects/*/configs/
• Loader & instantiator in detectron2/config/
• Supports default+user overrides (builder pattern)
b. Data Pipeline
• Dataset registration & catalog (detectron2/data/catalog.py)
• Built-in dataset scripts (COCO, LVIS, Cityscapes, PascalVOC)
• Samplers, BatchSamplers (distributed, grouped)
• Data transforms & augmentation modules
c. Model Zoo & Backbones
• detectron2/model_zoo: pre-defined config→URL mappings
• Backbones: ResNet, FPN, Swin, ViT, RegNet, MViT under modeling/backbone/
d. Meta-Architectures & Heads
• Meta-architectures: RCNN, RetinaNet, FCOS, Panoptic FPN, semantic seg
• Proposal generators: RPN, RRPN
e. Training Engine
• detectron2/engine/train_loop.py: main training loop with hooks (learning rate scheduling, checkpointing, logging)
• Solver: learning rate schedulers, optimizers under detectron2/solver/
• Checkpointing: detectron2/checkpoint/
f. Evaluation & Visualization
• COCO/LVIS/Pascal evaluators under detectron2/evaluation/
• Visualizer for predictions and Tensorboard events
g. Export & Deployment
• TorchScript and Caffe2 export modules under detectron2/export/
• C++/CUDA custom ops under detectron2/layers/csrc/
• Docker and “tools/deploy” wrappers for production
h. Auxiliary Utilities & Tools
• Logging, registry, file IO, comm utilities
• Stand-alone CLI tools: demo/demo.py, tools/*.py
• Project-specific extensions in projects/ (DensePose, DeepLab, PointRend, etc.)
Interactions & Data Flow
• User writes a config YAML → Configuration system builds:
• DataLoader ← dataset + transforms
• Model ← backbone + meta-arch + heads
• Optimizer & LR scheduler
• Training Loop: for each batch → Model forward → loss → backward → optimizer step → hook events (logging, checkpoint)
• Evaluation Loop: Model inference → evaluator collects metrics → visualizer outputs
• Export: Serialize model graph to TorchScript/Caffe2, including custom C++ kernels
Architectural Patterns & Principles
• Plugin Registry: Components (backbones, heads, datasets) register via decorators → extensibility
• Builder/Factory: Config-to-object instantiation for loose coupling
• Hook System: Training/evaluation behaviors decoupled from loop core
• Modular C++ extensions for performance-critical ops
Detail Level
• Essential custom C++ ops under layers/csrc
• High-level flow rather than every single YAML file