# hrnet/hrnet-semantic-segmentation architecture

> Architecture diagram of the GitHub repository hrnet/hrnet-semantic-segmentation, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-07-16). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/hrnet/hrnet-semantic-segmentation
- Repository: https://github.com/hrnet/hrnet-semantic-segmentation

## Overview

##### Project Type and Purpose
- This is an open-source deep learning framework for semantic image segmentation.
- It implements high-resolution networks (HRNet) augmented with Object Contextual Representation (OCR) and a Segmentation Transformer variant.
- Its main use cases are training and evaluating state-of-the-art segmentation models on datasets like Cityscapes, ADE20K, COCO-Stuff, PASCAL-Context, and LIP.

##### File-Structure Analysis
Top-level directories:
- data/       → dataset root, stratified by dataset name, with image and label subfolders plus split lists
- experiments/→ YAML configuration files per dataset and per model variant (HRNet / HRNet+OCR, Paddle vs. PyTorch)
- lib/        → core library modules, subdivided into:
– config/      : default and model-specific configuration definitions
– core/        : loss functions (criterion), training functions, distributed utilities
– datasets/    : dataset wrappers and PyTorch Dataset classes for each dataset
– models/      : neural network definitions (HRNet, segmentation heads, OCR head, sync-BN modules)
– utils/       : helper functions (distributed, logging, model summary, general utils)
- tools/      → high-level entry-points: train.py, test.py, and path setup
- run_local.sh, run_dist.sh → convenience scripts for single-machine and multi-GPU launches
- experiments/…/*.yaml → parameterized training/evaluation recipes
- requirements.txt → Python dependencies (PyTorch, etc.)

##### Architectural Insights from README
- The system is entirely driven by configuration files.
- Training and testing pipelines are launched via Python scripts (tools/train.py, tools/test.py), which parse a YAML and wire together: data loader → model → loss → optimizer → checkpointing → evaluation.
- Supports both PyTorch 0.4.1 and 1.1.0, with distributed training via torch.distributed.
- Uses sync-batch-norm (InplaceABN for PyTorch 0.4.1, native for PyTorch 1.1).
- Third-party integrations: PaddleCls pretrained weights via hubconf.py, detail-api for Pascal-Context preprocessing.

##### Components and Their Interactions
a. Main Components
- Data Ingestion Layer: data/ + lib/datasets/
- Configuration Module: lib/config/ + experiments/*.yaml
- Model Definitions: lib/models/ (HRNet, segmentation head, OCR head, sync-BN)
- Training Engine: tools/train.py orchestrating forward, backward, optimizer, scheduler
- Evaluation Engine: tools/test.py for inference, multi-scale/flip testing
- Distributed Infrastructure: torch.distributed, lib/utils/distributed.py, run_dist.sh
- External Services/Dependencies: PyTorch, PaddleCls, detail-api

b. Relationships and Data Flow
1. User selects an experiment YAML (experiments/…)
2. train.py loads config → instantiates Dataset → DataLoader
3. Configures model (HRNet base + optional OCR head) → loads pretrained weights (ImageNet, Paddle)
4. Wraps model for sync-BN and distributed training
5. Enters training loop:
– Batch read → forward pass → compute loss (lib/core/criterion.py)
– backward pass → optimizer.step → scheduler.step
– periodic checkpointing to disk
6. test.py performs inference: loads checkpoint → performs per-image or batched forward → applies multi-scale & flip augmentations if configured → computes metrics

c. Architectural Patterns and Principles
- Configuration-Driven Design: decouples code from hyperparameters/configs
- Modularization: clear separation of data, models, core algorithms, and utilities
- Extensibility: New datasets and models can be plugged in by adding modules under lib/
- Distributed Training Pattern: single-node multi-GPU via torch.distributed launcher
- Dependency Injection via hubconf.py for model registration

d. Key Technologies, Frameworks, Libraries
- Python 3.x, PyTorch 0.4.1 & 1.1.0
- YAML for experiment configs
- InplaceABN or PyTorch SyncBatchNorm for synchronized batch normalization
- PaddleCls for alternate pretrained backbones
- detail-api for PASCAL-Context data preprocessing

## Diagram (Mermaid)

```mermaid
flowchart TD
    DataStorage["Data Storage<br/>(data/)"]:::ondisk
    ConfigParser["Config Module<br/>(lib/config/ + experiments/)"]:::ondisk
    DatasetLoader["Dataset Loader<br/>(lib/datasets/)"]:::runtime
    ModelFactory["Model Factory<br/>(lib/models/)"]:::runtime
    SyncBN["Sync-BN Module<br/>(lib/models/sync_bn/)"]:::runtime
    TrainingLoop["Training Loop<br/>(tools/train.py)"]:::runtime
    EvalLoop["Evaluation Loop<br/>(tools/test.py)"]:::runtime
    CoreAlgo["Core Algorithms<br/>(lib/core/)"]:::runtime
    DistInfra["Distributed Infra<br/>(lib/utils/distributed.py)"]:::runtime
    Utilities["Utilities<br/>(lib/utils/)"]:::runtime
    Checkpoint["Checkpoint Storage"]:::ondisk
    Metrics["Metrics Output"]:::ondisk
    External["External Dependencies<br/>(PyTorch, PaddleCls, detail-api)"]:::external

    DataStorage -->|"raw images & labels"| DatasetLoader
    ConfigParser -->|"cfg options"| DatasetLoader
    DatasetLoader -->|"batches"| TrainingLoop

    ConfigParser -->|"cfg options"| ModelFactory
    ModelFactory -->|"model instance"| TrainingLoop
    ModelFactory -->|"model instance"| EvalLoop

    ModelFactory --> SyncBN
    SyncBN --> TrainingLoop

    TrainingLoop -->|"loss, grads"| CoreAlgo
    CoreAlgo --> TrainingLoop

    TrainingLoop -->|"save checkpoint"| Checkpoint
    EvalLoop -->|"load checkpoint"| Checkpoint
    EvalLoop -->|"compute metrics"| Metrics

    ConfigParser -->|"cfg options"| TrainingLoop
    ConfigParser -->|"cfg options"| EvalLoop

    TrainingLoop -->|"init dist"| DistInfra
    DistInfra --> TrainingLoop

    TrainingLoop -->|"helpers"| Utilities
    EvalLoop -->|"helpers"| Utilities

    TrainingLoop -->|"use ext libs"| External
    EvalLoop -->|"use ext libs"| External

    click DataStorage "https://github.com/hrnet/hrnet-semantic-segmentation/tree/HRNet-OCR/data/"
    click ConfigParser "https://github.com/hrnet/hrnet-semantic-segmentation/tree/HRNet-OCR/experiments/"
    click DatasetLoader "https://github.com/hrnet/hrnet-semantic-segmentation/tree/HRNet-OCR/lib/datasets/"
    click ModelFactory "https://github.com/hrnet/hrnet-semantic-segmentation/tree/HRNet-OCR/lib/models/"
    click SyncBN "https://github.com/hrnet/hrnet-semantic-segmentation/tree/HRNet-OCR/lib/models/sync_bn/"
    click TrainingLoop "https://github.com/hrnet/hrnet-semantic-segmentation/blob/HRNet-OCR/tools/train.py"
    click EvalLoop "https://github.com/hrnet/hrnet-semantic-segmentation/blob/HRNet-OCR/tools/test.py"
    click CoreAlgo "https://github.com/hrnet/hrnet-semantic-segmentation/blob/HRNet-OCR/lib/core/criterion.py"
    click DistInfra "https://github.com/hrnet/hrnet-semantic-segmentation/blob/HRNet-OCR/lib/utils/distributed.py"
    click Utilities "https://github.com/hrnet/hrnet-semantic-segmentation/blob/HRNet-OCR/lib/utils/utils.py"
    click External "https://github.com/hrnet/hrnet-semantic-segmentation/blob/HRNet-OCR/hubconf.py"

    classDef ondisk fill:#1f77b4,stroke:#000,color:#fff
    classDef runtime fill:#2ca02c,stroke:#000,color:#fff
    classDef external fill:#7f7f7f,stroke:#000,color:#fff
```
