# zhec/realtime_multi-person_pose_estimation architecture

> Architecture diagram of the GitHub repository zhec/realtime_multi-person_pose_estimation, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-03-31). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/zhec/realtime_multi-person_pose_estimation
- Repository: https://github.com/zhec/realtime_multi-person_pose_estimation

## Overview

##### Main Components
- Model Repository: The “model” folder holds pre-trained models (separated into COCO and MPI formats) with associated prototxt files. This is a key piece used for inference.
- Testing/Inference Components: The “testing” folder is organized into multiple subparts:
– Matlab-based testing (e.g., demo.m, various .m files, and configuration files in config.m) used for COCO evaluation.
– Python-based testing (with a Jupyter demo notebook in testing/python) where interactive exploration and demos can be executed.
– Supporting utilities, including JSON parser libraries (jsonlab) and visualization utilities (like export_fig).
- Training Pipeline: The “training” folder includes scripts and configuration files for preparing training data such as annotation processing (getANNO.m), mask generation (genCOCOMask.m), LMDB generation (genLMDB.py), and scripts that help set up prototxt configurations for training (setLayers.py). It also shows an example training workflow and loss curve generation through example_loss.

- Data Acquisition and Preprocessing:
– Start with the data pipeline (triggered by getData.sh) that downloads and extracts COCO data (images, annotations, and official toolbox data).
– Next, show the annotation conversion (getANNO.m), mask generation (genCOCOMask.m), and JSON creation (genJSON) as preprocessing steps that prepare the input for training.
- LMDB Generation and Network Configuration:
– Illustrate the use of genLMDB.py to produce a database that is then used by modified Caffe for training.
– The setLayers.py script creates necessary configuration (prototxt) files that define the network architecture.
- Training Component:
– Highlight that training is executed on a multi-GPU setup using custom shell scripts (e.g., train_pose.sh). The training initiates with a pre-trained VGG-19 model for the first 10 layers.
- Inference/Testing Component:
– For real-time testing, the system branches into various modalities: Matlab-based testing (via demo.m and other .m files), Python-based demos (through the Jupyter notebook), and a C++ variant (pointing towards the OpenPose project).
- External Dependencies:
– Note dependencies such as Caffe (for training and evaluation), Matlab, Python (and IPython/Jupyter), and external repositories like OpenPose for the C++ implementation.

##### Architectural Patterns and Design Principles
- Modular Design: The separation of concerns among training, testing, and model storage makes the system modular. This means that each component is responsible for a distinct part of the pipeline.
- External Service/Tool Integration: Include external tools such as the reference to OpenPose for real-time C++ testing.

## Diagram (Mermaid)

```mermaid
flowchart TD
    %% Data Acquisition & Preprocessing Layer
    subgraph "Data Acquisition & Preprocessing"
        A["COCO Data & Annotations (getData.sh)"]:::preprocessing
        B["Annotation Conversion (getANNO.m)"]:::preprocessing
        C["Mask Generation (genCOCOMask.m)"]:::preprocessing
        D["LMDB Generation (genLMDB.py)"]:::preprocessing
        E["LMDB Generation Shell (get_lmdb.sh)"]:::preprocessing
        F["Network Config Generation (setLayers.py)"]:::preprocessing
        G["Network Config Files (example_proto)"]:::preprocessing
    end

    %% Training Layer
    subgraph "Training Layer"
        H["Caffe Training (train_pose.sh)"]:::training
        I["Training Workflow & Loss (example_loss)"]:::training
        J["Model Repository"]:::model
    end

    %% Inference/Testing Layer
    subgraph "Inference Layer"
        K["Matlab Testing/Demo"]:::testing
        L["Python Notebook Demo"]:::testing
        O["C++ Inference (OpenPose)"]:::testing
        %% Utilities used in testing
        M["JSON Parser (jsonlab)"]:::testing
        N["Visualization Utility (export_fig)"]:::testing
    end

    %% Connections in Preprocessing
    A --> B
    B --> C
    C --> D
    D --> E
    E --> F
    F --> G

    %% Connections in Training
    G --> H
    H --> I
    I --> J

    %% Connections in Inference
    J --> K
    J --> L
    J --> O
    L --> M
    L --> N

    %% Click Events for Preprocessing Nodes
    click A "https://github.com/zhec/realtime_multi-person_pose_estimation/blob/master/training/getData.sh"
    click B "https://github.com/zhec/realtime_multi-person_pose_estimation/blob/master/training/getANNO.m"
    click C "https://github.com/zhec/realtime_multi-person_pose_estimation/blob/master/training/genCOCOMask.m"
    click D "https://github.com/zhec/realtime_multi-person_pose_estimation/blob/master/training/genLMDB.py"
    click E "https://github.com/zhec/realtime_multi-person_pose_estimation/blob/master/training/get_lmdb.sh"
    click F "https://github.com/zhec/realtime_multi-person_pose_estimation/blob/master/training/setLayers.py"
    click G "https://github.com/zhec/realtime_multi-person_pose_estimation/tree/master/training/example_proto"

    %% Click Events for Training Nodes
    click H "https://github.com/zhec/realtime_multi-person_pose_estimation/blob/master/training/example_proto/train_pose.sh"
    click I "https://github.com/zhec/realtime_multi-person_pose_estimation/tree/master/training/example_loss"
    click J "https://github.com/zhec/realtime_multi-person_pose_estimation/tree/master/model"

    %% Click Events for Inference Nodes
    click K "https://github.com/zhec/realtime_multi-person_pose_estimation/tree/master/testing"
    click L "https://github.com/zhec/realtime_multi-person_pose_estimation/tree/master/testing/python"
    click M "https://github.com/zhec/realtime_multi-person_pose_estimation/tree/master/testing/util/jsonlab"
    click N "https://github.com/zhec/realtime_multi-person_pose_estimation/tree/master/testing/util/ojwoodford-export_fig-5735e6d"

    %% Styles
    classDef preprocessing fill:#b3e5fc,stroke:#0288d1,stroke-width:2px;
    classDef training fill:#ffcc80,stroke:#f57c00,stroke-width:2px;
    classDef model fill:#c8e6c9,stroke:#388e3c,stroke-width:2px;
    classDef testing fill:#d1c4e9,stroke:#7b1fa2,stroke-width:2px;
```
