# arthur151/romp architecture

> Architecture diagram of the GitHub repository arthur151/romp, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-11-03). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/arthur151/romp
- Repository: https://github.com/arthur151/romp

## Overview

##### Project Type and Purpose
- This is a research-grade, Python-based deep learning library for monocular multi-person 3D human mesh recovery and tracking. It bundles three related CVPR/ICCV papers—ROMP (real-time mesh regression), BEV (depth relationships), and TRACE (temporal tracking with dynamic cameras)—into one repo. It supports training, evaluation, inference, export, and visualization, and exposes a simple cross-platform API via the `simple_romp` package.

##### High-Level File Structure
- Top-Level Directories
- configs/ – YAML files for various training/evaluation settings
- demo/ – example images and scripts
- docs/ – markdown guides on installation, training, evaluation, Docker, etc.
- romp/ – core implementation of ROMP & BEV (model definitions, dataset loaders, loss functions, training, evaluation, prediction, exports, visualization)
- trace/ – core implementation of TRACE (extensions for temporal regression under dynamic cameras)
- simple_romp/ – pip-installable wrapper for easy inference & export (fbx/glb/bvh) of ROMP, BEV, TRACE
- scripts/ – shell scripts to launch training, testing, exporting, Docker setup
- trace2/ under simple_romp – another wrapper for TRACE inference & evaluation
- Key Files
- `train.py`, `eval.py`, `predict/` – entry points for training, evaluation, inference
- `lib/models/`, `lib/dataset/`, `lib/loss_funcs/`, `lib/evaluation/`, `lib/visualization/` – modular subpackages implementing core stages
- `Dockerfile`, `docs/docker.md` – containerized environment setup

##### Main Architectural Components
a. Data Sources & Preprocessing
- Input modalities: images, video files, webcam streams
- Dataset modules under `*/lib/dataset/*` for handling COCO, 3DPW, MPII, Panoptic, video, custom challenge datasets
- Augmentations & preprocessing pipelines in `lib/utils/augments.py`, `lib/utils/*_utils.py`
b. Model Engine
- PyTorch-based backbones (ResNet50, HRNet32) and custom heads in `lib/models/`
- SMPL parameter regression modules in `lib/smpl_family/`
- BEV depth refinement and TRACE temporal modules (GRU/optical-flow/trackers)
c. Training Pipeline
- Entry point: `train.py`, `trace/train_video.py` under trace/
- Loss computation in `lib/loss_funcs/` (keypoints, parameters, prior, temporal)
- Multi-GPU support via `parallel/*` scripts
- Configuration via YAML files in configs/ and `lib/config.py`
d. Evaluation Pipeline
- Single/multi-dataset evaluation scripts in `eval.py`, `romp/eval.py`, `lib/evaluation/`
- Metrics for mesh accuracy (PCKh, MPJPE), 3D pose, tracking performance (HOTA, CLEAR)
e. Inference & Export
- Lightweight API wrapper in `simple_romp/` and `romp/predict/`—predict on image/video/webcam
- Exporters to FBX/GLB/BVH in `romp/exports/` and helper scripts (`export_fbx.sh`)
f. Visualization & Serving
- Open3D-based real-time visualizer with client/server in `lib/visualization/vis_platform/`
- Static renderers (vedo, matplotlib, Plotly)
g. Tracking (TRACE)
- Tracker modules in `trace/lib/tracker/`, `romp/lib/tracking/` with Kalman filters, data association
- Temporal regressors (GRU, Transformer) in `trace/lib/models/`

##### Relationships and Data Flow
- Data (images/video/webcam) → Dataset Loader → Preprocessing (resize, augment) → Model Forward → Predicted SMPL params + 3D mesh → (optional) Temporal Smoother/Tracker → Export/Save results → Visualization
- Training: add Loss → Backward → Optimizer update → Checkpoint
- Evaluation: Model checkpoint → Inference on test sets → Metric computation

##### Architectural Patterns & Design Principles
- Layered modular design: clear separation among Data, Model, Loss, Eval, Inference, Visualization
- Config-driven behavior: YAML files control all hyperparameters and dataset choices
- Wrapper API pattern: `simple_romp` hides internal complexity for end users
- Plug-in evaluators & exporters: easy to add new dataset or export format
- Client-server visualization: decouples rendering from inference

##### Detailed Elements
- Clear component names matching folder names (e.g., romp/lib/models, trace/lib/datasets)
- Notes on how YAML config files steer the pipelines
- Optional inset showing how `simple_romp` orchestrates the underlying `romp/` and `trace/` modules for end-user inference

## Diagram (Mermaid)

```mermaid
graph TD
    %% Data Sources & Preprocessing
    subgraph "Data Sources & Preprocessing"
        DS["Images/Video/Webcam"]:::green
        Dataset["Dataset Loaders\nromp/lib/dataset/"]:::green
        click Dataset "https://github.com/arthur151/romp/tree/master/romp/lib/dataset/"
        UtilsAug["Augmentations & Utils\nromp/lib/utils/augments.py"]:::green
        click UtilsAug "https://github.com/arthur151/romp/blob/master/romp/lib/utils/augments.py"
    end

    %% Core ROMP/BEV Engine
    subgraph "Core ROMP/BEV Engine"
        Models["Models\nromp/lib/models/"]:::blue
        click Models "https://github.com/arthur151/romp/tree/master/romp/lib/models/"
        SMPL["SMPL Regression\nromp/lib/smpl_family/"]:::blue
        click SMPL "https://github.com/arthur151/romp/tree/master/romp/lib/smpl_family/"
        Losses["Loss Functions\nromp/lib/loss_funcs/"]:::blue
        click Losses "https://github.com/arthur151/romp/tree/master/romp/lib/loss_funcs/"
        EvalROMP["Evaluation\nromp/lib/evaluation/"]:::blue
        click EvalROMP "https://github.com/arthur151/romp/tree/master/romp/lib/evaluation/"
        TrackROMP["Tracking\nromp/lib/tracking/"]:::blue
        click TrackROMP "https://github.com/arthur151/romp/tree/master/romp/lib/tracking/"
    end

    %% Core TRACE Engine
    subgraph "Core TRACE Engine"
        TModels["Temporal Models\ntrace/lib/models/"]:::blue
        click TModels "https://github.com/arthur151/romp/tree/master/trace/lib/models/"
        TDatasets["Datasets\ntrace/lib/datasets/"]:::blue
        click TDatasets "https://github.com/arthur151/romp/tree/master/trace/lib/datasets/"
        TLoss["Loss Functions\ntrace/lib/loss_funcs/"]:::blue
        click TLoss "https://github.com/arthur151/romp/tree/master/trace/lib/loss_funcs/"
        TEval["Evaluation\ntrace/lib/evaluation/"]:::blue
        click TEval "https://github.com/arthur151/romp/tree/master/trace/lib/evaluation/"
        EProp["EProPnP Solver\ntrace/lib/epropnp/"]:::blue
        click EProp "https://github.com/arthur151/romp/tree/master/trace/lib/epropnp/"
        TTracker["Tracking Module\ntrace/lib/tracker/"]:::blue
        click TTracker "https://github.com/arthur151/romp/tree/master/trace/lib/tracker/"
        TUti["Utils & Augmentations\ntrace/lib/utils/"]:::blue
        click TUti "https://github.com/arthur151/romp/tree/master/trace/lib/utils/"
    end

    %% Configurations
    subgraph "Configurations"
        Configs["YAML Configs\nconfigs/"]:::green
        click Configs "https://github.com/arthur151/romp/tree/master/configs/"
    end

    %% Entry-point Scripts
    subgraph "Entry-point Scripts"
        TrainROMP["ROMP Training/Eval\nromp/train.py"]:::orange
        click TrainROMP "https://github.com/arthur151/romp/blob/master/romp/train.py"
        TrainTrace["TRACE Training\ntrace/train_video.py"]:::orange
        click TrainTrace "https://github.com/arthur151/romp/blob/master/trace/train_video.py"
        Predict["Prediction\nromp/predict/"]:::orange
        click Predict "https://github.com/arthur151/romp/tree/master/romp/predict/"
    end

    %% Simple ROMP Wrapper API
    subgraph "Simple ROMP Wrapper API"
        SimpleAPI["simple_romp/"]:::purple
        click SimpleAPI "https://github.com/arthur151/romp/tree/master/simple_romp/"
    end

    %% Exporters & Visualization
    subgraph "Exporters & Visualization"
        Exports["ROMP Exports & Shell\nromp/exports/, scripts/"]:::red
        click Exports "https://github.com/arthur151/romp/blob/master/romp/exports/blender_mocap.py"
        click Exports "https://github.com/arthur151/romp/blob/master/scripts/export_fbx.sh"
        VisROMP["Visualization Server\nromp/lib/visualization/"]:::red
        click VisROMP "https://github.com/arthur151/romp/tree/master/romp/lib/visualization/"
        VisTrace["TRACE Visualization\ntrace/lib/visualization/"]:::red
        click VisTrace "https://github.com/arthur151/romp/blob/master/trace/lib/visualization/trajectory_viewer.py"
    end

    %% Data Flow
    DS --> Dataset --> Models --> SMPL --> Losses --> EvalROMP
    Dataset --> TDatasets --> TModels --> TLoss --> TEval
    Models --> SimpleAPI
    TDatasets --> SimpleAPI
    Configs --> TrainROMP --> Models
    Configs --> TrainTrace --> TModels
    TrainROMP --> Losses
    TrainTrace --> TLoss
    Predict --> SimpleAPI
    SimpleAPI --> Exports
    SimpleAPI --> VisROMP
    SimpleAPI --> VisTrace
    EvalROMP --> TEval

    %% Styles
    classDef green fill:#e0ffe0,stroke:#0a0,stroke-width:1px
    classDef blue fill:#e0f0ff,stroke:#00f,stroke-width:1px
    classDef orange fill:#fff0e0,stroke:#fa0,stroke-width:1px
    classDef purple fill:#f0e0ff,stroke:#a0f,stroke-width:1px
    classDef red fill:#ffe0e0,stroke:#f00,stroke-width:1px
```
