# vegetebird/mhformer architecture

> Architecture diagram of the GitHub repository vegetebird/mhformer, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-05-21). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/vegetebird/mhformer
- Repository: https://github.com/vegetebird/mhformer

## Overview

##### Project type and purpose
- This is a research‐grade, full-stack deep learning codebase for 3D human pose estimation called “MHFormer: Multi‐Hypothesis Transformer for 3D Human Pose Estimation.”
- The core functionality:
a) Load 2D keypoints (from ground truth or detector)
b) Feed sequences of 2D keypoints into a transformer‐based model (MHFormer)
c) Output 3D joint coordinates
d) Provide training, testing, and an in‐the‐wild demo pipeline using YOLOv3 and HRNet for 2D detection.

##### File structure
Top‐level organization:
- checkpoint/pretrained: stores downloaded pretrained weights
- common: data‐loading, skeleton definitions, utilities
- dataset: scripts and processed .npz data for Human3.6M
- model: MHFormer definition and sub‐modules (transformer, hypothesis generator)
- demo: live pipeline for videos (lib subfolder with YOLOv3 & HRNet detector code, video I/O, visualization)
- main.py: entrypoint for train/test
- requirements.txt: environment dependencies

Key patterns:
- Modular code (common / model / demo)
- Clear separation of data, model definition, and application scripts
- Use of PyTorch for model, YAML configs for detectors

##### README for additional insights
- Describes two workflows:
- Training / testing on Human3.6M dataset using main.py
- Demo pipeline: video → YOLOv3 human detection → HRNet keypoint extraction → MHFormer 3D estimation → visualization
- Lists external dependencies (PyTorch, torchvision, sorted detector, pretrained weights)
- Points to directory conventions: dataset/, checkpoint/, demo/video/

## Diagram (Mermaid)

```mermaid
flowchart TB
    subgraph "Training & Evaluation Pipeline"
        DataStore_H36M(("Human3.6M Dataset\n(npz files)")):::data
        LoadData["load_data_hm36.py"]:::data
        H36MDataset["h36m_dataset.py"]:::data
        DataStore_H36M --> LoadData
        LoadData --> H36MDataset
        H36MDataset --> Core
    end

    subgraph "Model Modules"
        Core["MHFormer Core\n(PyTorch 1.7.1)"]:::core
        subgraph "Transformer Modules"
            Trans["trans.py"]:::core
            Hypothesis["trans_hypothesis.py"]:::core
        end
        Core --> Trans
        Core --> Hypothesis
    end

    Core --> Loss
    Loss["Loss & Backprop"]:::utils
    Loss --> CheckpointStore

    subgraph "Checkpoint & Config"
        CheckpointStore(("Checkpoint Store")):::config
        Config["main.py CLI\n(argparse)"]:::utils
        Config --> Core
        Config --> Loss
        Config --> CheckpointStore
    end

    subgraph "Demo Pipeline"
        VideoFile(("Sample Video\n(mp4)")):::data
        subgraph "YOLOv3 Detector"
            YPre["preprocess.py"]:::demo
            YDark["darknet.py"]:::demo
            YDet["human_detector.py"]:::demo
            YPre --> YDark --> YDet
        end
        subgraph "HRNet Detector"
            HRGen["gen_kpts.py"]:::demo
            HRConfig["default.py\n(models.py)"]:::demo
            HRModel["pose_hrnet.py"]:::demo
            HRGen --> HRModel
        end
        VideoFile --> YPre
        YDet --> HRGen
        HRModel --> Core
        Core --> Vis
    end

    Vis["vis.py"]:::demo

    subgraph "Utilities"
        U1["utils.py"]:::utils
        U2["opt.py"]:::utils
        U3["generator.py"]:::utils
        U4["skeleton.py"]:::utils
        U5["camera.py"]:::utils
    end
    U1 --> LoadData
    U1 --> Core

    classDef data fill:#9f6,stroke:#333,stroke-width:1px
    classDef core fill:#6cf,stroke:#333,stroke-width:1px
    classDef demo fill:#f90,stroke:#333,stroke-width:1px
    classDef utils fill:#ccc,stroke:#333,stroke-width:1px
    classDef config fill:#999,stroke:#333,stroke-width:1px

    click LoadData "https://github.com/vegetebird/mhformer/blob/main/common/load_data_hm36.py"
    click H36MDataset "https://github.com/vegetebird/mhformer/blob/main/common/h36m_dataset.py"
    click Core "https://github.com/vegetebird/mhformer/blob/main/model/mhformer.py"
    click Trans "https://github.com/vegetebird/mhformer/blob/main/model/module/trans.py"
    click Hypothesis "https://github.com/vegetebird/mhformer/blob/main/model/module/trans_hypothesis.py"
    click U1 "https://github.com/vegetebird/mhformer/blob/main/common/utils.py"
    click U2 "https://github.com/vegetebird/mhformer/blob/main/common/opt.py"
    click U3 "https://github.com/vegetebird/mhformer/blob/main/common/generator.py"
    click U4 "https://github.com/vegetebird/mhformer/blob/main/common/skeleton.py"
    click U5 "https://github.com/vegetebird/mhformer/blob/main/common/camera.py"
    click Config "https://github.com/vegetebird/mhformer/blob/main/main.py"
    click CheckpointStore "https://github.com/vegetebird/mhformer/tree/main/checkpoint/pretrained/"
    click YPre "https://github.com/vegetebird/mhformer/blob/main/demo/lib/yolov3/preprocess.py"
    click YDark "https://github.com/vegetebird/mhformer/blob/main/demo/lib/yolov3/darknet.py"
    click YDet "https://github.com/vegetebird/mhformer/blob/main/demo/lib/yolov3/human_detector.py"
    click HRGen "https://github.com/vegetebird/mhformer/blob/main/demo/lib/hrnet/gen_kpts.py"
    click HRConfig "https://github.com/vegetebird/mhformer/blob/main/demo/lib/hrnet/lib/config/default.py"
    click HRModel "https://github.com/vegetebird/mhformer/blob/main/demo/lib/hrnet/lib/models/pose_hrnet.py"
    click Vis "https://github.com/vegetebird/mhformer/blob/main/demo/vis.py"
    click VideoFile "https://github.com/vegetebird/mhformer/blob/main/demo/video/sample_video.mp4"
```
