How hrnet/hrnet-semantic-segmentation works
Project Type and Purpose
• This is an open-source deep learning framework for semantic image segmentation.
• It implements high-resolution networks (HRNet) augmented with Object Contextual Representation (OCR) and a Segmentation Transformer variant.
• Its main use cases are training and evaluating state-of-the-art segmentation models on datasets like Cityscapes, ADE20K, COCO-Stuff, PASCAL-Context, and LIP.
File-Structure Analysis
Top-level directories:
• data/ → dataset root, stratified by dataset name, with image and label subfolders plus split lists
• experiments/→ YAML configuration files per dataset and per model variant (HRNet / HRNet+OCR, Paddle vs. PyTorch)
• lib/ → core library modules, subdivided into:
– config/ : default and model-specific configuration definitions
– core/ : loss functions (criterion), training functions, distributed utilities
– datasets/ : dataset wrappers and PyTorch Dataset classes for each dataset
– models/ : neural network definitions (HRNet, segmentation heads, OCR head, sync-BN modules)
– utils/ : helper functions (distributed, logging, model summary, general utils)
• tools/ → high-level entry-points: train.py, test.py, and path setup
• run_local.sh, run_dist.sh → convenience scripts for single-machine and multi-GPU launches
• experiments/…/*.yaml → parameterized training/evaluation recipes
• requirements.txt → Python dependencies (PyTorch, etc.)
Architectural Insights from README
• The system is entirely driven by configuration files.
• Training and testing pipelines are launched via Python scripts (tools/train.py, tools/test.py), which parse a YAML and wire together: data loader → model → loss → optimizer → checkpointing → evaluation.
• Supports both PyTorch 0.4.1 and 1.1.0, with distributed training via torch.distributed.
• Uses sync-batch-norm (InplaceABN for PyTorch 0.4.1, native for PyTorch 1.1).
• Third-party integrations: PaddleCls pretrained weights via hubconf.py, detail-api for Pascal-Context preprocessing.
Components and Their Interactions
a. Main Components
• Data Ingestion Layer: data/ + lib/datasets/
• Configuration Module: lib/config/ + experiments/*.yaml
• Model Definitions: lib/models/ (HRNet, segmentation head, OCR head, sync-BN)
• Training Engine: tools/train.py orchestrating forward, backward, optimizer, scheduler
• Evaluation Engine: tools/test.py for inference, multi-scale/flip testing
• Distributed Infrastructure: torch.distributed, lib/utils/distributed.py, run_dist.sh
• External Services/Dependencies: PyTorch, PaddleCls, detail-api
b. Relationships and Data Flow
1. User selects an experiment YAML (experiments/…)
2. train.py loads config → instantiates Dataset → DataLoader
3. Configures model (HRNet base + optional OCR head) → loads pretrained weights (ImageNet, Paddle)
4. Wraps model for sync-BN and distributed training
5. Enters training loop:
– Batch read → forward pass → compute loss (lib/core/criterion.py)
– backward pass → optimizer.step → scheduler.step
– periodic checkpointing to disk
6. test.py performs inference: loads checkpoint → performs per-image or batched forward → applies multi-scale & flip augmentations if configured → computes metrics
c. Architectural Patterns and Principles
• Configuration-Driven Design: decouples code from hyperparameters/configs
• Modularization: clear separation of data, models, core algorithms, and utilities
• Extensibility: New datasets and models can be plugged in by adding modules under lib/
• Distributed Training Pattern: single-node multi-GPU via torch.distributed launcher
• Dependency Injection via hubconf.py for model registration
d. Key Technologies, Frameworks, Libraries
• Python 3.x, PyTorch 0.4.1 & 1.1.0
• YAML for experiment configs
• InplaceABN or PyTorch SyncBatchNorm for synchronized batch normalization
• PaddleCls for alternate pretrained backbones
• detail-api for PASCAL-Context data preprocessing