How facebookresearch/convnext works
Project Type and Purpose
• This is a research‐grade PyTorch codebase implementing ConvNeXt for:
a. Image classification on ImageNet (main pipeline)
b. Downstream object detection (using MMDetection)
c. Downstream semantic segmentation (using MMSegmentation)
• It provides training, evaluation, pre‐trained weights, and config‐driven workflows for all three tasks.
Core Modules & File‐Structure Analysis
• Root Scripts:
• main.py – entry point for classification training/eval
• engine.py – training/evaluation loop (losses, metrics, logging)
• datasets.py – dataset definitions & transformations (ImageNet, CIFAR, etc.)
• optim_factory.py – constructs optimizer & LR schedulers
• run_with_submitit.py – wrapper for distributed/slurm jobs
• utils.py – misc helpers (logging, checkpointing, seed setting)
• Model Definitions:
• models/convnext.py – core ConvNeXt blocks & variants (Tiny/Small/Base/Large)
• models/convnext_isotropic.py – isotropic variant
• Object Detection: object_detection/
• configs/… – MMDetection config hierarchy (base → ConvNeXt‐backboned Mask R‑CNN/Cascade Mask R‑CNN)
• mmcv_custom/… – custom hooks (text logging), optimizer constructors, checkpoint runners
• mmdet/… – vendored or extended MMDetection backbone definitions
• Semantic Segmentation: semantic_segmentation/
• configs/… – MMSegmentation config hierarchy (base → UPerNet w/ ConvNeXt backbones)
• mmcv_custom/… – custom Apex runner, text logging, optimizer constructors
• backbone/convnext.py – segmentation‐specific ConvNeXt modifications
High‐Level System Design Components
A. Entry Point (main.py or task‐specific scripts)
– Reads CLI args (model, data path, distributed setup)
– Loads configuration (via argument parsing + Python config files)
– Initializes logging, seeds, device (CPU/GPU/Distributed)
– Dispatches to classification or downstream task pipeline
B. Data Layer (datasets.py + config files)
– Defines DataLoader, data augmentations, dataset splits
– Task‐specific data readers (ImageNet, COCO, ADE20K) via config
C. Model Layer
– Core ConvNeXt architecture in models/
– Task‐specific “backbone” wrappers (in object_detection/mmdet and semantic_segmentation/backbone)
– Heads: classification head, detection heads (RPN, ROI), segmentation head
D. Training Engine (engine.py)
– Single‐/multi‐GPU training loop: forward, loss, backward, optimizer step, logging
– Evaluation loop: validation metrics, checkpoint saving
E. Optimization & Scheduling (optim_factory.py)
– Builds AdamW/Layer‐decay optimizers for ConvNeXt blocks
– Learning‐rate schedulers (cosine, step) as defined in config
F. Distributed & Cluster Execution (run_with_submitit.py)
– Wraps main.py to launch jobs on Slurm or multi‐node via Submitit
– Handles checkpoint resume & environment setup
G. Utilities (utils.py + mmcv_custom)
– Logging, checkpoint abstractions, custom checkpoint runners, text output, mixed‐precision support
Inter‐Component Relationships & Data Flow
• main.py → parses args → loads config → calls “run_pipeline”
• run_pipeline → calls dataset builder → model builder → engine.train/evaluate
• engine ← uses: model, optimizer (optim_factory), dataloader
• engine → logs via utils and optionally mmcv_custom runner → checkpoint via utils
• For downstream tasks: main.py references object_detection.configs or semantic_segmentation.configs
which register models and pipelines in MMDetection/MMSeg frameworks
• run_with_submitit wraps the entire flow for distributed scheduling
Architectural Patterns & Design Principles
• Modular tasks separation: classification vs. detection vs. segmentation
• Config‐driven workflows: Python config files under each task’s configs/ folder
• Plugin/extension of external frameworks (MMDetection, MMSegmentation) via mmcv_custom
• Separation of concerns: dataset, model, engine, optimizer, utils each in its own module
• Reuse of ConvNeXt core across all pipelines