# pixelite1201/camerahmr architecture

> Architecture diagram of the GitHub repository pixelite1201/camerahmr, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-05-24). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/pixelite1201/camerahmr
- Repository: https://github.com/pixelite1201/camerahmr

## Overview

##### Project type and purpose
- CameraHMR is a research‐oriented full‐stack machine‐learning system for 3D human mesh recovery under perspective projection.
- It provides end-to-end pipelines for training, evaluation, demo inference, data fetching, and a post-processing module (CamSMPLify) for optimization‐based refinement.

##### File structure
- Top-level CLI/front-end scripts:
- demo.py, demo_keypoints.py, eval.py, mesh_estimator.py, dataset_vis.py
- train.py, train_keypoints.py
- download and fetch scripts under scripts/ and CamSMPLify/download_data.sh
- Core library (core/):
- core/backbones: vision backbones (ViT, HRNet)
- core/cam_model: camera-aware model definitions (backbone wrappers, FL-Net)
- core/components: transformer modules, MLP heads
- core/heads: SMPL/SMPL-X regression heads
- core/models & trainers: camerahmr_model.py, camerahmr_trainer.py, densekp_model.py, densekp_trainer.py
- core/datasets: PyTorch data modules, dataset splits, keypoint utilities
- core/utils: rendering helpers, geometry, evaluation, logging, Detectron2 integration
- core/configs_hydra: centralized experiment configuration (data, model, trainer, paths)
- CamSMPLify module (CamSMPLify/): optimization-based SMPL alignment
- Miscellaneous files: requirements.txt, ReadMe.md, teaser assets, docs/

##### README for additional insights
- Confirms the use of PyTorch, Hydra for configuration, Detectron2, ViTPose, SMPLify
- Outlines multiple entry points: demo, training, evaluation, data download
- References external data sources (4D-Humans, BEDLAM) and how to fetch them

##### Main components and their interactions
a. Data ingestion
- Image folders or webdatasets → core/datasets → DataLoader
- Optional keypoint extractor (DenseKP or OpenPose)
b. Model backbone
- Vision encoder (ViT or HRNet) in core/backbones
c. Feature processing
- PoseTransformer & t_cond_MLP in core/components
d. SMPL parameter regression
- SMPL/SMPL-X heads in core/heads
e. Loss calculation
- core/losses.py and CamSMPLify/losses.py for training and post-processing
f. Training/Evaluation orchestration
- core/camerahmr_trainer.py, core/densekp_trainer.py
- Hydra configurations control optimizer, scheduler, distributed settings
g. Rendering & visualization
- core/utils/renderer_cam.py, renderer_pyrd.py, CamSMPLify/utils/renderer_cam.py
h. CLI tools
- demo.py, train.py, eval.py, dataset_vis.py
i. Post-processing
- CamSMPLify.optimize → refinement of SMPL parameters via optimization

##### Architectural patterns and design principles
- Modular design: clear separation of data, model, loss, training, rendering
- Configuration‐driven experiments via Hydra (config hierarchy)
- Pluggable backbones and heads for easy swapping
- CLI entry points orchestrate high-level workflows
- Utility layer isolates visualization, logging, data extraction

- Clear component labels (e.g., “Backbone: ViT/HRNet”, “SMPL Head”)
- Data sources vs models vs utils vs post-processing
- Training vs inference paths
- Callouts for configuration management (Hydra) and external services (4D-Humans data source)

## Diagram (Mermaid)

```mermaid
flowchart TB
    %% Layer 1: User Interaction (CLI)
    subgraph "1. User Interaction (CLI)"
        direction TB
        Demo["demo.py"]:::cli
        KeypointDemo["demo_keypoints.py"]:::cli
        Eval["eval.py"]:::cli
        MeshEst["mesh_estimator.py"]:::cli
        DataVis["dataset_vis.py"]:::cli
        Train["train.py"]:::cli
        TrainKP["train_keypoints.py"]:::cli
    end

    %% Layer 2: Configuration (Hydra)
    subgraph "2. Configuration (Hydra)"
        direction TB
        ConfigCamHMR["camerahmr.yaml"]:::config
        ConfigDenseKP["densekp.yaml"]:::config
        ConfigTrain["train.yaml"]:::config
        ConfigKPTrain["densekp_train.yaml"]:::config
        ConfigEval["eval.yaml"]:::config
        ConfigPaths["paths/default.yaml"]:::config
        ConfigHydra["hydra/default.yaml"]:::config
    end

    %% Layer 3: Data Layer
    subgraph "3. Data Layer"
        direction TB
        Dataset["dataset.py"]:::data
        DatasetTrain["dataset_train.py"]:::data
        DatasetVal["dataset_val.py"]:::data
        DatasetKP["dataset_keypoints.py"]:::data
        DataModuleKP["datamodule_keypoints.py"]:::data
        DataUtils["utils.py"]:::data
        Fetch4D["fetch_4dhumans_training_labels.sh"]:::data
        FetchBedlam["fetch_bedlam_training_data.sh"]:::data
        FetchDemo["fetch_demo_data.sh"]:::data
        FetchTest["fetch_test_labels.sh"]:::data
        FetchCSData["download_data.sh"]:::data
    end

    %% Layer 4: Model Layer
    subgraph "4. Model Layer"
        direction TB
        subgraph Backbones
            direction TB
            ViT["vit.py"]:::model
            HRNet["hrnet.py"]:::model
        end
        FLNet["fl_net.py"]:::model
        PoseTrans["pose_transformer.py"]:::model
        TMLP["t_cond_mlp.py"]:::model
        subgraph Heads
            direction TB
            SMPLCliff["smpl_head_cliff.py"]:::model
            SMPLKP["smpl_head_keypoints.py"]:::model
            SMPLXCliff["smplx_head_cliff.py"]:::model
        end
        CamHMRModel["camerahmr_model.py"]:::model
        DenseKPModel["densekp_model.py"]:::model
    end

    %% Layer 5: Training/Eval Layer
    subgraph "5. Training/Eval"
        direction TB
        CamHMRTrainer["camerahmr_trainer.py"]:::train
        DenseKPTrainer["densekp_trainer.py"]:::train
        LossTrain["losses.py"]:::train
        LossPost["CamSMPLify/losses.py"]:::train
    end

    %% Layer 6: Post-Processing (CamSMPLify)
    subgraph "6. Post-Processing (CamSMPLify)"
        direction TB
        CSMain["cam_smplify.py"]:::post
        CSOpt["optimize.py"]:::post
        CSConst["constants.py"]:::post
        CSImgUtils["image_utils.py"]:::post
        CSRenderer["renderer_cam.py"]:::post
        CSSmplOP["smpl_openpose.py"]:::post
    end

    %% Layer 7: Visualization & Utils
    subgraph "7. Visualization & Utilities"
        direction TB
        RendererCam["renderer_cam.py"]:::viz
        RendererPyrd["renderer_pyrd.py"]:::viz
        Geometry["geometry.py"]:::viz
        EvalUtils["eval_utils.py"]:::viz
        DetectronUtils["utils_detectron2.py"]:::viz
        PyLogger["pylogger.py"]:::viz
        TrainUtils["train_utils.py"]:::viz
        MiscUtils["misc.py"]:::viz
        RichUtils["rich_utils.py"]:::viz
    end

    %% Data Flow & Relationships
    Demo -->|reads| ConfigCamHMR
    KeypointDemo -->|reads| ConfigDenseKP
    Train -->|reads| ConfigCamHMR
    TrainKP -->|reads| ConfigDenseKP
    Eval -->|reads| ConfigEval

    ConfigCamHMR -->|feeds| Dataset
    ConfigDenseKP -->|feeds| DatasetKP

    Dataset -->|provides| CamHMRModel
    DatasetKP -->|provides| DenseKPModel

    CamHMRModel --> PoseTrans
    CamHMRModel --> TMLP
    CamHMRModel --> SMPLCliff
    DenseKPModel --> SMPLKP

    ViT --> CamHMRModel
    HRNet --> CamHMRModel
    FLNet --> CamHMRModel

    CamHMRModel --> CamHMRTrainer
    DenseKPModel --> DenseKPTrainer
    LossTrain --> CamHMRTrainer
    LossTrain --> DenseKPTrainer

    CamHMRTrainer -->|outputs SMPL| RendererCam
    DenseKPTrainer -->|outputs Keypoints| RendererPyrd

    RendererCam -->|visualize| EvalUtils
    RendererPyrd -->|visualize| EvalUtils

    CamHMRModel -->|SMPL output| CSMain
    CSMain --> CSOpt
    CSOpt --> CSRenderer
    CSOpt --> CSSmplOP
    CSRenderer --> RendererCam

    %% Click Events
    click Demo "https://github.com/pixelite1201/camerahmr/blob/master/demo.py"
    click KeypointDemo "https://github.com/pixelite1201/camerahmr/blob/master/demo_keypoints.py"
    click Eval "https://github.com/pixelite1201/camerahmr/blob/master/eval.py"
    click MeshEst "https://github.com/pixelite1201/camerahmr/blob/master/mesh_estimator.py"
    click DataVis "https://github.com/pixelite1201/camerahmr/blob/master/dataset_vis.py"
    click Train "https://github.com/pixelite1201/camerahmr/blob/master/train.py"
    click TrainKP "https://github.com/pixelite1201/camerahmr/blob/master/train_keypoints.py"
    click Fetch4D "https://github.com/pixelite1201/camerahmr/blob/master/scripts/fetch_4dhumans_training_labels.sh"
    click FetchBedlam "https://github.com/pixelite1201/camerahmr/blob/master/scripts/fetch_bedlam_training_data.sh"
    click FetchDemo "https://github.com/pixelite1201/camerahmr/blob/master/scripts/fetch_demo_data.sh"
    click FetchTest "https://github.com/pixelite1201/camerahmr/blob/master/scripts/fetch_test_labels.sh"
    click FetchCSData "https://github.com/pixelite1201/camerahmr/blob/master/CamSMPLify/download_data.sh"
    click ViT "https://github.com/pixelite1201/camerahmr/blob/master/core/backbones/vit.py"
    click HRNet "https://github.com/pixelite1201/camerahmr/blob/master/core/cam_model/backbone/hrnet.py"
    click FLNet "https://github.com/pixelite1201/camerahmr/blob/master/core/cam_model/fl_net.py"
    click PoseTrans "https://github.com/pixelite1201/camerahmr/blob/master/core/components/pose_transformer.py"
    click TMLP "https://github.com/pixelite1201/camerahmr/blob/master/core/components/t_cond_mlp.py"
    click SMPLCliff "https://github.com/pixelite1201/camerahmr/blob/master/core/heads/smpl_head_cliff.py"
    click SMPLKP "https://github.com/pixelite1201/camerahmr/blob/master/core/heads/smpl_head_keypoints.py"
    click SMPLXCliff "https://github.com/pixelite1201/camerahmr/blob/master/core/heads/smplx_head_cliff.py"
    click CamHMRModel "https://github.com/pixelite1201/camerahmr/blob/master/core/camerahmr_model.py"
    click CamHMRTrainer "https://github.com/pixelite1201/camerahmr/blob/master/core/camerahmr_trainer.py"
    click DenseKPModel "https://github.com/pixelite1201/camerahmr/blob/master/core/densekp_model.py"
    click DenseKPTrainer "https://github.com/pixelite1201/camerahmr/blob/master/core/densekp_trainer.py"
    click Dataset "https://github.com/pixelite1201/camerahmr/blob/master/core/datasets/dataset.py"
    click DatasetTrain "https://github.com/pixelite1201/camerahmr/blob/master/core/datasets/dataset_train.py"
    click DatasetVal "https://github.com/pixelite1201/camerahmr/blob/master/core/datasets/dataset_val.py"
    click DatasetKP "https://github.com/pixelite1201/camerahmr/blob/master/core/datasets/dataset_keypoints.py"
    click DataModuleKP "https://github.com/pixelite1201/camerahmr/blob/master/core/datasets/datamodule_keypoints.py"
    click DataUtils "https://github.com/pixelite1201/camerahmr/blob/master/core/datasets/utils.py"
    click LossTrain "https://github.com/pixelite1201/camerahmr/blob/master/core/losses.py"
    click LossPost "https://github.com/pixelite1201/camerahmr/blob/master/CamSMPLify/losses.py"
    click CSMain "https://github.com/pixelite1201/camerahmr/blob/master/CamSMPLify/cam_smplify.py"
    click CSOpt "https://github.com/pixelite1201/camerahmr/blob/master/CamSMPLify/optimize.py"
    click CSConst "https://github.com/pixelite1201/camerahmr/blob/master/CamSMPLify/constants.py"
    click CSImgUtils "https://github.com/pixelite1201/camerahmr/blob/master/CamSMPLify/utils/image_utils.py"
    click CSRenderer "https://github.com/pixelite1201/camerahmr/blob/master/CamSMPLify/utils/renderer_cam.py"
    click CSSmplOP "https://github.com/pixelite1201/camerahmr/blob/master/CamSMPLify/utils/smpl_openpose.py"
    click ConfigCamHMR "https://github.com/pixelite1201/camerahmr/blob/master/core/configs_hydra/experiment/camerahmr.yaml"
    click ConfigDenseKP "https://github.com/pixelite1201/camerahmr/blob/master/core/configs_hydra/experiment/densekp.yaml"
    click ConfigTrain "https://github.com/pixelite1201/camerahmr/blob/master/core/configs_hydra/data/train.yaml"
    click ConfigKPTrain "https://github.com/pixelite1201/camerahmr/blob/master/core/configs_hydra/data/densekp_train.yaml"
    click ConfigEval "https://github.com/pixelite1201/camerahmr/blob/master/core/configs_hydra/data/eval.yaml"
    click ConfigPaths "https://github.com/pixelite1201/camerahmr/blob/master/core/configs_hydra/paths/default.yaml"
    click ConfigHydra "https://github.com/pixelite1201/camerahmr/blob/master/core/configs_hydra/hydra/default.yaml"

    %% Styles
    classDef cli fill:#FFDFBA,stroke:#E69A00,color:#333
    classDef config fill:#D0E7FF,stroke:#1C70FF,color:#333
    classDef data fill:#E2FFD7,stroke:#2CA02C,color:#333
    classDef model fill:#FFE2F0,stroke:#D62728,color:#333
    classDef train fill:#FFF2B8,stroke:#BCBD22,color:#333
    classDef post fill:#ECC3FF,stroke:#9467BD,color:#333
    classDef viz fill:#F0F0F0,stroke:#7F7F7F,color:#333
```
