# shubham-goel/4d-humans architecture

> Architecture diagram of the GitHub repository shubham-goel/4d-humans, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-05-02). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/shubham-goel/4d-humans
- Repository: https://github.com/shubham-goel/4d-humans

## Overview

##### Project type and purpose
- This is a research‐oriented, full-pipeline 3D human reconstruction and tracking system (4DHumans/HMR2.0).
- It provides:
- A demo/inference API (images & videos)
- A training pipeline (data download, Hydra‐configured experiments, PyTorch‐Lightning)
- An evaluation suite for multiple benchmarks
- A simple Gradio front-end

##### File structure
- Top‐level scripts:
- demo.py (image mode inference)
- track.py (video tracking via PHALP)
- gradio_app.py (UI)
- train.py (training entry point)
- eval.py (evaluation entry point)
- Core python package: `hmr2/`
- configs & configs_hydra (Hydra‐based experiment/config management)
- datasets (data loaders, preprocessors)
- models (backbones, transformer components, heads, losses)
- utils (download, geometry, rendering, logging)
- Deployment/build: environment.yml, setup.py

##### README for additional insights
- Uses: PyTorch, PyTorch-Lightning, Hydra, Detectron2 (ViTDet), PHALP, SMPL model
- Workflow stages: data fetch, preprocessing, training, inference/demo, evaluation
- Outputs: .obj meshes, side/front renders, tracked 3D pose files

## Diagram (Mermaid)

```mermaid
flowchart TB
    subgraph "User Interface Layer"
        demo["demo.py"]:::ui
        track["track.py"]:::ui
        train["train.py"]:::ui
        eval["eval.py"]:::ui
        gradio["gradio_app.py"]:::ui
    end

    subgraph "Configuration Layer"
        hydra["hmr2/configs_hydra/"]:::config
        legacy["hmr2/configs/"]:::config
    end

    subgraph "Data Layer"
        example_data["example_data/"]:::data
        fetch["fetch_training_data.sh"]:::data
        datasets["hmr2/datasets/"]:::data
    end

    subgraph "Compute/Model Layer"
        model_entry["hmr2/models/hmr2.py"]:::compute
        backbone_vit["hmr2/models/backbones/vit.py"]:::compute
        backbone_vitpose["hmr2/models/backbones/vit_vitpose.py"]:::compute
        transformer["hmr2/models/components/pose_transformer.py"]:::compute
        mlp["hmr2/models/components/t_cond_mlp.py"]:::compute
        smpl_head["hmr2/models/heads/smpl_head.py"]:::compute
        smpl_wrapper["hmr2/models/smpl_wrapper.py"]:::compute
        losses["hmr2/models/losses.py"]:::compute
        discriminator["hmr2/models/discriminator.py"]:::compute
    end

    subgraph "Training Abstraction"
        trainer_cfg["hmr2/configs_hydra/trainer/"]:::config
        pylogger["hmr2/utils/pylogger.py"]:::util
        richutils["hmr2/utils/rich_utils.py"]:::util
    end

    subgraph "Postprocessing & Output Layer"
        mesh_renderer["hmr2/utils/mesh_renderer.py"]:::util
        renderer["hmr2/utils/renderer.py"]:::util
        openpose["hmr2/utils/render_openpose.py"]:::util
        skel_renderer["hmr2/utils/skeleton_renderer.py"]:::util
        geometry["hmr2/utils/geometry.py"]:::util
        texture["hmr2/utils/texture_utils.py"]:::util
        pck["hmr2/utils/pck_accuracy.py"]:::util
        poseutils["hmr2/utils/pose_utils.py"]:::util
    end

    subgraph "Utilities & Support"
        download["hmr2/utils/download.py"]:::util
        detectron2utils["hmr2/utils/utils_detectron2.py"]:::util
        misc["hmr2/utils/misc.py"]:::util
    end

    subgraph "Deployment & Packaging"
        env["environment.yml"]:::storage
        setup["setup.py"]:::storage
        license["LICENSE.md"]:::storage
        readme["README.md"]:::storage
    end

    subgraph "External Services"
        phalp["PHALP Repository"]:::external
        smpl["SMPL Model Download"]:::external
    end

    demo -->|"reads configs"| hydra
    track -->|"reads configs"| hydra
    train -->|"reads configs"| hydra
    eval -->|"reads configs"| hydra
    gradio -->|"reads configs"| hydra

    hydra -->|"provides configs to"| datasets
    legacy -->|"legacy configs to"| datasets

    example_data -->|"provides samples"| datasets
    fetch -->|"downloads data"| example_data
    datasets -->|"loads data into"| model_entry

    model_entry --> backbone_vit
    model_entry --> backbone_vitpose
    model_entry --> transformer
    model_entry --> mlp
    model_entry --> smpl_head
    model_entry --> smpl_wrapper
    model_entry --> losses
    model_entry --> discriminator

    trainer_cfg -->|"configures trainer"| train
    train -->|"trains model"| model_entry
    train -->|"logs & checkpoints to"| env
    pylogger -->|used by| train
    richutils -->|used by| train

    model_entry -->|"inference output to"| mesh_renderer
    mesh_renderer -->|"renders to"| env
    renderer -->|"renders views to"| env
    openpose -->|"passes skeleton to"| skel_renderer
    skel_renderer -->|"renders skeleton to"| env
    geometry -->|utilized by| mesh_renderer
    texture -->|utilized by| mesh_renderer

    eval -->|"evaluates via metrics"| pck
    eval -->|"evaluates via metrics"| poseutils

    detectron2utils -->|helpers for| model_entry
    download -->|"fetches SMPL assets from"| smpl

    track -->|"calls PHALP API"| phalp
    track -->|"uses inference"| model_entry

    demo -->|"runs inference"| model_entry
    gradio -->|"UI for inference"| model_entry

    click demo "https://github.com/shubham-goel/4d-humans/blob/main/demo.py"
    click track "https://github.com/shubham-goel/4d-humans/blob/main/track.py"
    click train "https://github.com/shubham-goel/4d-humans/blob/main/train.py"
    click eval "https://github.com/shubham-goel/4d-humans/blob/main/eval.py"
    click gradio "https://github.com/shubham-goel/4d-humans/blob/main/gradio_app.py"
    click hydra "https://github.com/shubham-goel/4d-humans/tree/main/hmr2/configs_hydra/"
    click legacy "https://github.com/shubham-goel/4d-humans/tree/main/hmr2/configs/"
    click example_data "https://github.com/shubham-goel/4d-humans/tree/main/example_data/"
    click fetch "https://github.com/shubham-goel/4d-humans/blob/main/fetch_training_data.sh"
    click datasets "https://github.com/shubham-goel/4d-humans/tree/main/hmr2/datasets/"
    click model_entry "https://github.com/shubham-goel/4d-humans/blob/main/hmr2/models/hmr2.py"
    click backbone_vit "https://github.com/shubham-goel/4d-humans/blob/main/hmr2/models/backbones/vit.py"
    click backbone_vitpose "https://github.com/shubham-goel/4d-humans/blob/main/hmr2/models/backbones/vit_vitpose.py"
    click transformer "https://github.com/shubham-goel/4d-humans/blob/main/hmr2/models/components/pose_transformer.py"
    click mlp "https://github.com/shubham-goel/4d-humans/blob/main/hmr2/models/components/t_cond_mlp.py"
    click smpl_head "https://github.com/shubham-goel/4d-humans/blob/main/hmr2/models/heads/smpl_head.py"
    click smpl_wrapper "https://github.com/shubham-goel/4d-humans/blob/main/hmr2/models/smpl_wrapper.py"
    click losses "https://github.com/shubham-goel/4d-humans/blob/main/hmr2/models/losses.py"
    click discriminator "https://github.com/shubham-goel/4d-humans/blob/main/hmr2/models/discriminator.py"
    click trainer_cfg "https://github.com/shubham-goel/4d-humans/tree/main/hmr2/configs_hydra/trainer/"
    click pylogger "https://github.com/shubham-goel/4d-humans/blob/main/hmr2/utils/pylogger.py"
    click richutils "https://github.com/shubham-goel/4d-humans/blob/main/hmr2/utils/rich_utils.py"
    click mesh_renderer "https://github.com/shubham-goel/4d-humans/blob/main/hmr2/utils/mesh_renderer.py"
    click renderer "https://github.com/shubham-goel/4d-humans/blob/main/hmr2/utils/renderer.py"
    click openpose "https://github.com/shubham-goel/4d-humans/blob/main/hmr2/utils/render_openpose.py"
    click skel_renderer "https://github.com/shubham-goel/4d-humans/blob/main/hmr2/utils/skeleton_renderer.py"
    click geometry "https://github.com/shubham-goel/4d-humans/blob/main/hmr2/utils/geometry.py"
    click texture "https://github.com/shubham-goel/4d-humans/blob/main/hmr2/utils/texture_utils.py"
    click pck "https://github.com/shubham-goel/4d-humans/blob/main/hmr2/utils/pck_accuracy.py"
    click poseutils "https://github.com/shubham-goel/4d-humans/blob/main/hmr2/utils/pose_utils.py"
    click download "https://github.com/shubham-goel/4d-humans/blob/main/hmr263/utils/download.py"
    click detectron2utils "https://github.com/shubham-goel/4d-humans/blob/main/hmr2/utils/utils_detectron2.py"
    click misc "https://github.com/shubham-goel/4d-humans/blob/main/hmr2/utils/misc.py"
    click env "https://github.com/shubham-goel/4d-humans/blob/main/environment.yml"
    click setup "https://github.com/shubham-goel/4d-humans/blob/main/setup.py"

    classDef ui fill:#f2f2f2,stroke:#333,stroke-width:1px
    classDef config fill:#ffcc00,stroke:#333,stroke-width:1px
    classDef data fill:#bbeeff,stroke:#333,stroke-width:1px
    classDef compute fill:#a8e6cf,stroke:#333,stroke-width:1px
    classDef util fill:#d9d9d9,stroke:#333,stroke-width:1px
    classDef storage fill:#99ccff,stroke:#333,stroke-width:1px
    classDef external fill:#e6ccff,stroke:#333,stroke-width:1px
```
