# hrnet/higherhrnet-human-pose-estimation architecture

> Architecture diagram of the GitHub repository hrnet/higherhrnet-human-pose-estimation, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-05-22). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/hrnet/higherhrnet-human-pose-estimation
- Repository: https://github.com/hrnet/higherhrnet-human-pose-estimation

## Overview

##### Project type and purpose
- **Type**: A research‐oriented machine learning codebase for bottom‐up multi‐person human pose estimation.
- **Purpose**: Train and evaluate the HigherHRNet model on COCO and CrowdPose datasets, using mixed‐precision and distributed training, with a clear, config‐driven pipeline.

##### File structure
- **Top‐level folders**
- `lib/` – core library code (config parsing, model definitions, dataset loaders, training/inference logic, utilities).
- `experiments/` – YAML config files for different datasets and model variants.
- `tools/` – CLI scripts for training, validation, dataset preparation.
- `figures/` – static architecture illustrations.
- **Supporting files**
- `requirements.txt`, `.gitignore`, `LICENSE`, `README.md`
- `models/`, `data/`, `output/`, `log/` (created at runtime for pretrained weights, raw data, outputs, tensorboard logs).

##### README for insights
- The README describes a **config‐driven** pipeline (YAML → model/dataset instantiation).
- Dependencies: Python 3.6, PyTorch ≥1.1.0, NVIDIA Apex (for FP16), COCOAPI, CrowdPoseAPI.
- Usage patterns: single‐/multi‐GPU training, FP16, SyncBN, multi‐scale testing.

##### Main components and their interactions
a. **Configuration Loader** (`lib/config`)
– Reads YAML files under `experiments/...`
– Exposes a global config object (`default.py`, `models.py`)
b. **Dataset & Data Pipeline** (`lib/dataset`)
– `build.py` factory builds dataset instances (COCO, CrowdPose)
– `transforms/` apply augmentations
– `target_generators/` create heatmaps/offset targets
– Feeds DataLoader to the trainer/inference engine
c. **Model Definition** (`lib/models/pose_higher_hrnet.py`)
– Backbone: HRNet (multi‐resolution feature extractor)
– HigherHRNet head: feature pyramid via transposed convolutions
d. **Training Core** (`lib/core/trainer.py`, `loss.py`)
– Builds optimizer (supports FP16 via `lib/fp16_utils`)
– Computes multi‐scale supervision losses
– Checkpointing and logging to `output/` and `log/`
e. **Inference Core** (`lib/core/inference.py`)
– Runs forward pass, aggregates multi‐resolution predictions
– Post‐processing to decode keypoints
f. **Utilities** (`lib/utils`)
– Visualization (`vis.py`), I/O (`zipreader.py`), general helpers (`utils.py`)
g. **CLI Tools** (`tools/`)
– `dist_train.py` – distributed training entrypoint
– `valid.py` – validation/testing script
– `crowdpose_concat_train_val.py` – data prep
– `_init_paths.py` – path setup

##### Architectural patterns and design principles
- **Separation of Concerns**: config, data, model, training, inference, utils are modular.
- **Factory/Builder**: dataset and model objects are instantiated via builders reading the same config.
- **Config‐Driven**: behavior entirely governed by YAML files (no hard‐coded parameters).
- **Extensibility**: adding a new dataset or model variant only requires adding a new class and updating the config.
- **Distributed & Mixed Precision**: uses PyTorch DDP and NVIDIA Apex for scalable training.

## Diagram (Mermaid)

```mermaid
flowchart TD
    %% Configuration Layer
    subgraph "Configuration Layer"
        style ConfigurationLayer fill:#e0f7fa,stroke:#006064
        ExpConfig["experiments/*.yaml"]:::configFile
        ConfigModule["lib/config"]:::module
    end

    %% Data Layer
    subgraph "Data Layer"
        style DataLayer fill:#e8f5e9,stroke:#1b5e20
        DatasetBuilder["lib/dataset/build.py"]:::module
        subgraph "Dataset Classes"
            COCODataset["COCODataset"]:::module
            CrowdPoseDataset["CrowdPoseDataset"]:::module
        end
        subgraph "Data Transforms"
            TransformsBuild["build.py"]:::module
            Transforms["transforms.py"]:::module
        end
        TargetGenerators["target_generators.py"]:::module
    end

    %% Compute Layer
    subgraph "Compute Layer"
        style ComputeLayer fill:#fff3e0,stroke:#e65100
        ModelDef["pose_higher_hrnet.py"]:::module
        subgraph "Training Core"
            Trainer["trainer.py"]:::module
            Loss["loss.py"]:::module
        end
        subgraph "Mixed-Precision Utils"
            FP16Opt["fp16_optimizer.py"]:::module
            FP16Util["fp16util.py"]:::module
            LossScaler["loss_scaler.py"]:::module
        end
        DistSupport["group.py"]:::module
        Inference["inference.py"]:::module
    end

    %% Execution Layer
    subgraph "Execution Layer"
        style ExecLayer fill:#f3e5f5,stroke:#4a148c
        ToolsDistTrain["dist_train.py"]:::script
        ToolsValid["valid.py"]:::script
        ToolsPrep["crowdpose_concat_train_val.py"]:::script
        InitPaths["_init_paths.py"]:::script
    end

    %% Utilities
    subgraph "Utilities"
        style UtilsLayer fill:#eceff1,stroke:#37474f
        Utils["utils.py"]:::module
        Vis["vis.py"]:::module
        ZipReader["zipreader.py"]:::module
        TransformsUtil["transforms.py"]:::module
    end

    %% Data Stores
    dataRaw[(data/)]:::dataStore
    modelsStore[(models/)]:::dataStore
    outputStore[(output/)]:::dataStore
    logStore[(log/)]:::dataStore

    %% External Services
    PyTorch[(PyTorch)]:::external
    Apex[(NVIDIA Apex)]:::external
    COCOAPI[(COCOAPI)]:::external
    CrowdAPI[(CrowdPoseAPI)]:::external
    GPUs[(GPUs)]:::external

    %% Flow Connections
    ExpConfig -->|"YAML → parameters"| ConfigModule
    ConfigModule -->|"config object"| DatasetBuilder

    DatasetBuilder -->|"builds"| COCODataset
    DatasetBuilder -->|"builds"| CrowdPoseDataset

    COCODataset -->|"images + annotations"| TransformsBuild
    CrowdPoseDataset -->|"images + annotations"| TransformsBuild

    TransformsBuild -->|"applies"| Transforms
    Transforms -->|"augmented data"| TargetGenerators
    TargetGenerators -->|"targets"| Trainer

    %% DataLoader implied: builder -> Trainer
    DatasetBuilder -->|"DataLoader"| Trainer

    Trainer -->|"compute loss"| Loss
    Trainer -->|"model forward"| ModelDef
    Trainer -->|"uses FP16"| FP16Opt
    Trainer -->|"uses FP16"| FP16Util
    Trainer -->|"uses FP16"| LossScaler
    Trainer -->|"distributed"| DistSupport

    Trainer -->|"checkpoints"| modelsStore
    Trainer -->|"logs"| logStore

    Inference -->|"forward pass"| ModelDef
    Inference -->|"data from"| DatasetBuilder
    Inference -->|"predictions"| outputStore

    ToolsDistTrain -->|"invokes"| Trainer
    ToolsValid -->|"invokes"| Inference
    ToolsPrep -->|"prepares data"| dataRaw
    InitPaths -->|"sets paths"| ToolsDistTrain

    %% External dependencies
    Trainer -->|"runs on"| GPUs
    Trainer -->|"uses"| PyTorch
    Trainer -->|"uses"| Apex
    Inference -->|"uses"| PyTorch
    Inference -->|"uses"| Apex
    COCODataset -->|"API"| COCOAPI
    CrowdPoseDataset -->|"API"| CrowdAPI

    %% Click Events
    click ConfigModule "https://github.com/hrnet/higherhrnet-human-pose-estimation/blob/master/lib/config/default.py"
    click ConfigModule "https://github.com/hrnet/higherhrnet-human-pose-estimation/blob/master/lib/config/models.py"
    click ExpConfig "https://github.com/hrnet/higherhrnet-human-pose-estimation/blob/master/experiments/coco/higher_hrnet/*.yaml"
    click ExpConfig "https://github.com/hrnet/higherhrnet-human-pose-estimation/blob/master/experiments/crowd_pose/higher_hrnet/*.yaml"
    click DatasetBuilder "https://github.com/hrnet/higherhrnet-human-pose-estimation/blob/master/lib/dataset/build.py"
    click COCODataset "https://github.com/hrnet/higherhrnet-human-pose-estimation/blob/master/lib/dataset/COCODataset.py"
    click CrowdPoseDataset "https://github.com/hrnet/higherhrnet-human-pose-estimation/blob/master/lib/dataset/CrowdPoseDataset.py"
    click TargetGenerators "https://github.com/hrnet/higherhrnet-human-pose-estimation/blob/master/lib/dataset/target_generators/target_generators.py"
    click TransformsBuild "https://github.com/hrnet/higherhrnet-human-pose-estimation/blob/master/lib/dataset/transforms/build.py"
    click Transforms "https://github.com/hrnet/higherhrnet-human-pose-estimation/blob/master/lib/dataset/transforms/transforms.py"
    click ModelDef "https://github.com/hrnet/higherhrnet-human-pose-estimation/blob/master/lib/models/pose_higher_hrnet.py"
    click Trainer "https://github.com/hrnet/higherhrnet-human-pose-estimation/blob/master/lib/core/trainer.py"
    click Loss "https://github.com/hrnet/higherhrnet-human-pose-estimation/blob/master/lib/core/loss.py"
    click FP16Opt "https://github.com/hrnet/higherhrnet-human-pose-estimation/blob/master/lib/fp16_utils/fp16_optimizer.py"
    click FP16Util "https://github.com/hrnet/higherhrnet-human-pose-estimation/blob/master/lib/fp16_utils/fp16util.py"
    click LossScaler "https://github.com/hrnet/higherhrnet-human-pose-estimation/blob/master/lib/fp16_utils/loss_scaler.py"
    click Inference "https://github.com/hrnet/higherhrnet-human-pose-estimation/blob/master/lib/core/inference.py"
    click DistSupport "https://github.com/hrnet/higherhrnet-human-pose-estimation/blob/master/lib/core/group.py"
    click Utils "https://github.com/hrnet/higherhrnet-human-pose-estimation/blob/master/lib/utils/utils.py"
    click Vis "https://github.com/hrnet/higherhrnet-human-pose-estimation/blob/master/lib/utils/vis.py"
    click ZipReader "https://github.com/hrnet/higherhrnet-human-pose-estimation/blob/master/lib/utils/zipreader.py"
    click TransformsUtil "https://github.com/hrnet/higherhrnet-human-pose-estimation/blob/master/lib/utils/transforms.py"
    click ToolsDistTrain "https://github.com/hrnet/higherhrnet-human-pose-estimation/blob/master/tools/dist_train.py"
    click ToolsValid "https://github.com/hrnet/higherhrnet-human-pose-estimation/blob/master/tools/valid.py"
    click ToolsPrep "https://github.com/hrnet/higherhrnet-human-pose-estimation/blob/master/tools/crowdpose_concat_train_val.py"
    click InitPaths "https://github.com/hrnet/higherhrnet-human-pose-estimation/blob/master/tools/_init_paths.py"

    %% Styles
    classDef module fill:#bbdefb,stroke:#0d47a1,color:#0d47a1
    classDef configFile fill:#c8e6c9,stroke:#1b5e20,color:#1b5e20
    classDef script fill:#ffe082,stroke:#ff6f00,color:#ff6f00
    classDef dataStore fill:#cfd8dc,stroke:#455a64,color:#455a64
    classDef external fill:#eeeeee,stroke:#9e9e9e,color:#616161
```
