# sanweiliti/rohm architecture

> Architecture diagram of the GitHub repository sanweiliti/rohm, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-04-13). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/sanweiliti/rohm
- Repository: https://github.com/sanweiliti/rohm

## Overview

##### Project Type and Purpose
- The project is a research/experimental full-stack implementation for human motion reconstruction, specifically using diffusion models to recover complete and plausible motion sequences from noisy and occluded inputs.
- Key components include two separate sub-models (PoseNet for local motion and TrajNet for global trajectory) and a diffusion module that underpins both. The README highlights training, evaluation, and testing on multiple datasets (AMASS, PROX, EgoBody), which all bring unique input preparation and configuration steps.

##### Analysis of the File Structure
- The top-level directories clearly separate configuration (cfg_files) from core functionalities:
- cfg_files contains YAML configuration files for evaluation, testing, and training. These files dictate how the modules are set up in different experiment scenarios.
- data_loaders holds data ingestion and preprocessing logic, critical for converting raw input (AMASS, PROX, EgoBody) into the desired format.
- diffusion includes modules for different aspects of the diffusion process (e.g., gaussian_diffusion for both PoseNet and TrajNet, resample and respace utilities, logging, and fp16 support).
- model contains implementations of the core networks (posenet, trajnet) and components like heads and configuration samplers.
- train includes training loops for both PoseNet and TrajNet, showing the progression (curriculum training) from initial training stages to fine-tuning.
- Utilities (in utils) support various tasks such as seed fixing, occlusion mask generation, rendering, and general model utilities.
- There is also clear segregation for test and evaluation scripts that direct the flow for different datasets and scenarios.

##### High-Level System Components to Capture
a. Input/Preprocessing:
- Data Preparation: Preprocessing scripts (preprocessing_amass.py) used for preparing AMASS data.
- Data Loaders: Modules in the data_loaders folder that import and process data (common routines, quaternion handling, etc.).
- External Data: Downloaded checkpoints, SMPL-X model data, and dataset folders (data/ and datasets/ directories) mentioned in the README.

b. Core Model Components:
- PoseNet: Located in model/posenet.py and trained via train_posenet.py; responsible for local motion reconstruction.
- TrajNet: Found in model/trajnet.py and trained with train_trajnet.py; handles the global trajectory reconstruction.
- Conditioning Module: As described in the README (and implied by the coupling between PoseNet and TrajNet models), this module integrates the outputs of both branches to ensure consistency.
- Diffusion Module: In diffusion/ (including gaussian_diffusion_posenet.py and gaussian_diffusion_trajnet.py) which encapsulates the iterative inference method central to the reconstruction.

c. Training, Testing, and Evaluation Pipelines:
- Training Loops: Scripts in the train directory that outline the training stages; curriculum learning is emphasized with stage-specific behavior.
- Testing Framework: Test scripts (e.g., test_amass_full.py, test_prox_egobody.py) that run inference experiments on various datasets.
- Evaluation Pipeline: Evaluation scripts (e.g., eval_amass_full.py, eval_prox_egobody.py) that compute metrics and generate visuals/rendered outputs.

d. Utility and Support Modules:
- Utility Components: Helper functions in utils (e.g., model_util.py, render_util.py, vis_util.py) that support debugging, configuration, and rendering.
- Diffusion Support: Additional utilities for FP16 computation and logging within the diffusion directory.

##### Relationships and Data Flow
▸ The TrajNet branch (global trajectory reconstruction) which includes its diffusion process and training loop.
▸ The PoseNet branch (local motion reconstruction) following a similar pattern.
- Both branches then converge at the Conditioning Module, which ensures the two sub-tasks are coherent.
- From the Conditioning Module, flow forward to Output Generation (reconstructed motion sequences) and final Evaluation (metrics, visualization, rendering).

##### Detailed Element Inclusions
- Two Processing Branches:
▸ Left branch: Global Trajectory Reconstruction (TrajNet, its associated diffusion module, and training loops).
▸ Right branch: Local Pose Reconstruction (PoseNet, its associated diffusion module, and training loops).
- Utility Libraries and External Tool Dependencies: Clearly denote where tools from external repositories support parts of the process (e.g., CLIFF for initialization).

## Diagram (Mermaid)

```mermaid
flowchart TD
    %% Data Ingestion Block
    subgraph "Data Ingestion"
        DS["Datasets (AMASS, PROX, EgoBody)"]:::external
        PRE["Preprocessing (AMASS)"]:::process
        DL["Data Loaders"]:::process
        DS -->|"preprocess"| PRE
        PRE -->|"load"| DL
    end

    %% PoseNet Pipeline
    subgraph "PoseNet Pipeline"
        PN["PoseNet Model"]:::model
        PN_train["Training Script (PoseNet)"]:::train
        PN_loop["Training Loop (PoseNet)"]:::train
        PN_diff["Diffusion (PoseNet)"]:::diffusion
        DL -->|"feeds"| PN
        PN -->|"calls"| PN_train
        PN_train -->|"executes"| PN_loop
        PN_loop -->|"utilizes"| PN_diff
    end

    %% TrajNet Pipeline
    subgraph "TrajNet Pipeline"
        TT["TrajNet Model"]:::model
        TT_train["Training Script (TrajNet)"]:::train
        TT_loop["Training Loop (TrajNet)"]:::train
        TT_diff["Diffusion (TrajNet)"]:::diffusion
        DL -->|"feeds"| TT
        TT -->|"calls"| TT_train
        TT_train -->|"executes"| TT_loop
        TT_loop -->|"utilizes"| TT_diff
    end

    %% Both pipelines converge at the Conditioning Module
    PN_diff -->|"integrates with"| CM["Conditioning Module"]:::fusion
    TT_diff -->|"integrates with"| CM

    %% Evaluation and Output Generation
    CM -->|"generates"| EV["Evaluation & Visualization\n(test & eval scripts)"]:::eval
    EV -->|"produces"| OUT["Output Generation"]:::output

    %% Utility and Configuration Support (external support)
    UTIL["Utility Modules"]:::utility
    CONF["Configuration Files"]:::config
    EV ---|"uses"| UTIL
    EV ---|"configured by"| CONF

    %% Click Events
    click DL "https://github.com/sanweiliti/rohm/tree/main/data_loaders/"
    click PRE "https://github.com/sanweiliti/rohm/blob/main/preprocessing_amass.py"
    click PN "https://github.com/sanweiliti/rohm/blob/main/model/posenet.py"
    click PN_train "https://github.com/sanweiliti/rohm/blob/main/train_posenet.py"
    click PN_loop "https://github.com/sanweiliti/rohm/blob/main/train/training_loop_posenet.py"
    click TT "https://github.com/sanweiliti/rohm/blob/main/model/trajnet.py"
    click TT_train "https://github.com/sanweiliti/rohm/blob/main/train_trajnet.py"
    click TT_loop "https://github.com/sanweiliti/rohm/blob/main/train/training_loop_trajnet.py"
    click PN_diff "https://github.com/sanweiliti/rohm/blob/main/diffusion/gaussian_diffusion_posenet.py"
    click TT_diff "https://github.com/sanweiliti/rohm/blob/main/diffusion/gaussian_diffusion_trajnet.py"
    click CM "https://github.com/sanweiliti/rohm/blob/main/model/heads.py"
    click UTIL "https://github.com/sanweiliti/rohm/tree/main/utils/"
    click CONF "https://github.com/sanweiliti/rohm/tree/main/cfg_files/"

    %% Styles
    classDef external fill:#AED6F1,stroke:#1B4F72,stroke-width:2px;
    classDef process fill:#D5F5E3,stroke:#196F3D,stroke-width:2px;
    classDef model fill:#F9E79F,stroke:#B7950B,stroke-width:2px;
    classDef train fill:#E8DAEF,stroke:#6C3483,stroke-width:2px;
    classDef diffusion fill:#FAD7A0,stroke:#D35400,stroke-width:2px;
    classDef fusion fill:#D6DBDF,stroke:#566573,stroke-width:2px;
    classDef eval fill:#FCF3CF,stroke:#B9770E,stroke-width:2px;
    classDef output fill:#D1F2EB,stroke:#117864,stroke-width:2px;
    classDef utility fill:#F5CBA7,stroke:#9A7D0A,stroke-width:2px;
    classDef config fill:#D6EAF8,stroke:#2471A3,stroke-width:2px;
```
