# hybridrobotics/whole_body_tracking architecture

> Architecture diagram of the GitHub repository hybridrobotics/whole_body_tracking, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-08-17). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/hybridrobotics/whole_body_tracking
- Repository: https://github.com/hybridrobotics/whole_body_tracking

## Overview

##### Project Type & Purpose
- This is neither a traditional web app nor a pure library—it’s an end-to-end RL motion-tracking pipeline.
- Core purpose: preprocess human motion data, host motions in WandB, run physics-based simulation (IsaacSim/IsaacLab), train/test RL policies (PPO via RSL-RL), and log artifacts.

##### High-Level Components
A. Data Ingestion & Processing
- Input: raw .csv motion datasets (e.g. LAFAN1).
- scripts/csv_to_npz.py → forward-kinematics npz files.
- Upload npz artifacts to WandB registry.
B. Simulation Environment
- IsaacSim v4.5.0 + IsaacLab v2.1.0 (external service).
- Uses robot descriptions (source/.../assets) and MDP definitions.
C. Reinforcement Learning Core
- Library package (source/whole_body_tracking/whole_body_tracking):
- tasks/tracking/mdp: commands, rewards, observations, terminations, events
- tasks/tracking/tracking_env_cfg.py (env config)
- config/g1 & config/humanoid (agent hyperparams)
- robots/ (actuator models, joint parameters)
- utils/ exporter & my_on_policy_runner (training loop)
D. Orchestration Scripts
- scripts/rsl_rl/train.py (launches training job, logger=wandb)
- scripts/rsl_rl/play.py (policy evaluation/replay)
- scripts/replay_npz.py (quick sim visualization)
- scripts/upload_npz.py (alternative upload)
E. External Services & Artifacts
- WandB registry for storing/loading motions & run logs
- GCS (robot description tarball)
- Conda/Python environment (packaged via pyproject.toml & setup.py)

##### Interactions & Data Flow
- Raw CSV motion → (csv_to_npz) → NPZ artifact → WandB registry
- train.py:
- Pull NPZ from WandB → initialize simulation MDP → run PPO (my_on_policy_runner) → log metrics & model artifacts back to WandB
- play.py / replay_npz.py:
- Pull NPZ or trained policy from WandB → launch IsaacSim → render trajectories
- Core library modules (mdp, config, robots) are loaded at runtime by scripts.

##### Architectural Patterns & Principles
- Modular separation:
- Data preprocessing (scripts) vs. core algorithm (library) vs. orchestration (train/play scripts).
- Plugin/config pattern:
- TOML configs under tasks/tracking/config drive env & agent parameters.
- Dependency injection:
- CLI args select tasks (e.g. “Tracking-Flat-G1-v0”), wandb paths, headless mode.

## Diagram (Mermaid)

```mermaid
flowchart TD
    %% Data Layer
    subgraph "Data Layer"
        DP["Motion Preprocessor<br/>(csv_to_npz)"]:::code
        NPZ["NPZ Artifact"]:::data
        UP["NPZ Upload Script"]:::code
    end

    %% Orchestration Scripts
    subgraph "Orchestration Layer"
        TRAIN["Training Orchestration<br/>(train.py)"]:::code
        PLAY["Policy Evaluation/Playback<br/>(play.py)"]:::code
        REPLAY["Quick Simulation Replay<br/>(replay_npz.py)"]:::code
    end

    %% Core RL Library
    subgraph "RL Core Library" 
        CoreRoot["Library Root"]:::code
        subgraph "Task MDP Definitions"
            MDP["MDP Modules"]:::code
        end
        TEC["Env Configuration<br/>(tracking_env_cfg.py)"]:::code
        subgraph "Agent & Env Configs"
            G1R["G1 PPO Config"]:::code
            G1E["G1 Env Config"]:::code
            HUMR["Humanoid PPO Config"]:::code
            HUME["Humanoid Env Config"]:::code
        end
        ROB["Robot Models & Params"]:::code
        ASSET["Simulation Assets"]:::code
        subgraph "Utilities"
            EXP["Exporter"]:::code
            RUN["On-Policy Runner"]:::code
        end
    end

    %% External Services
    subgraph "External Services"
        WB["WandB Registry"]:::external
        SIM["IsaacSim / IsaacLab"]:::external
        GCS["GCS Storage"]:::external
    end

    %% Packaging & CI
    subgraph "Packaging & CI"
        EXT["Extension Config<br/>(extension.toml)"]:::config
        PYP["pyproject.toml"]:::config
        SETUP["setup.py"]:::config
        FLAKE[".flake8"]:::config
        PRE[".pre-commit-config.yaml"]:::config
    end

    %% Data Flow
    RAW["Raw CSV Motion Data"]:::data -->|csv_to_npz| DP
    DP -->|np.savez| NPZ
    NPZ -->|wandb.Artifact.log| WB
    UP -->|wandb.Artifact.log| WB

    %% Training Flow
    WB -->|wandb.Artifact.get| TRAIN
    TRAIN -->|import core| CoreRoot
    CoreRoot -->|log metrics & model| WB

    %% Playback Flow
    WB -->|wandb.Artifact.get| PLAY
    WB -->|wandb.Artifact.get| REPLAY
    PLAY -->|launch| SIM
    REPLAY -->|launch| SIM

    %% Core Dependencies
    CoreRoot --> MDP
    CoreRoot --> TEC
    CoreRoot --> G1R
    CoreRoot --> G1E
    CoreRoot --> HUMR
    CoreRoot --> HUME
    CoreRoot --> ROB
    CoreRoot --> ASSET
    CoreRoot --> EXP
    CoreRoot --> RUN

    %% External Storage
    ROB -->|downloads assets| GCS
    ASSET -->|hosted on| GCS

    %% Packaging & CI dependencies
    CoreRoot --> EXT
    CoreRoot --> PYP
    CoreRoot --> SETUP
    CoreRoot --> FLAKE
    CoreRoot --> PRE

    %% Click Events
    click DP "https://github.com/hybridrobotics/whole_body_tracking/blob/main/scripts/csv_to_npz.py"
    click UP "https://github.com/hybridrobotics/whole_body_tracking/blob/main/scripts/upload_npz.py"
    click REPLAY "https://github.com/hybridrobotics/whole_body_tracking/blob/main/scripts/replay_npz.py"
    click TRAIN "https://github.com/hybridrobotics/whole_body_tracking/blob/main/scripts/rsl_rl/train.py"
    click PLAY "https://github.com/hybridrobotics/whole_body_tracking/blob/main/scripts/rsl_rl/play.py"
    click CoreRoot "https://github.com/hybridrobotics/whole_body_tracking/blob/main/source/whole_body_tracking/whole_body_tracking/__init__.py"
    click MDP "https://github.com/hybridrobotics/whole_body_tracking/tree/main/source/whole_body_tracking/whole_body_tracking/tasks/tracking/mdp"
    click TEC "https://github.com/hybridrobotics/whole_body_tracking/blob/main/source/whole_body_tracking/whole_body_tracking/tasks/tracking/tracking_env_cfg.py"
    click G1R "https://github.com/hybridrobotics/whole_body_tracking/blob/main/source/whole_body_tracking/whole_body_tracking/tasks/tracking/config/g1/agents/rsl_rl_ppo_cfg.py"
    click G1E "https://github.com/hybridrobotics/whole_body_tracking/blob/main/source/whole_body_tracking/whole_body_tracking/tasks/tracking/config/g1/flat_env_cfg.py"
    click HUMR "https://github.com/hybridrobotics/whole_body_tracking/blob/main/source/whole_body_tracking/whole_body_tracking/tasks/tracking/config/humanoid/agents/rsl_rl_ppo_cfg.py"
    click HUME "https://github.com/hybridrobotics/whole_body_tracking/blob/main/source/whole_body_tracking/whole_body_tracking/tasks/tracking/config/humanoid/flat_env_cfg.py"
    click ROB "https://github.com/hybridrobotics/whole_body_tracking/tree/main/source/whole_body_tracking/whole_body_tracking/robots"
    click ASSET "https://github.com/hybridrobotics/whole_body_tracking/tree/main/source/whole_body_tracking/whole_body_tracking/assets"
    click EXP "https://github.com/hybridrobotics/whole_body_tracking/blob/main/source/whole_body_tracking/whole_body_tracking/utils/exporter.py"
    click RUN "https://github.com/hybridrobotics/whole_body_tracking/blob/main/source/whole_body_tracking/whole_body_tracking/utils/my_on_policy_runner.py"
    click EXT "https://github.com/hybridrobotics/whole_body_tracking/blob/main/source/whole_body_tracking/config/extension.toml"
    click PYP "https://github.com/hybridrobotics/whole_body_tracking/blob/main/pyproject.toml"
    click SETUP "https://github.com/hybridrobotics/whole_body_tracking/blob/main/source/whole_body_tracking/setup.py"
    click FLAKE "https://github.com/hybridrobotics/whole_body_tracking/blob/main/.flake8"
    click PRE "https://github.com/hybridrobotics/whole_body_tracking/blob/main/.pre-commit-config.yaml"

    %% Styles
    classDef code fill:#D0E8FF,stroke:#005B99,color:#000;
    classDef external fill:#DFFFE0,stroke:#339900,color:#000;
    classDef data fill:#FFF3BF,stroke:#FFB800,color:#000;
    classDef config fill:#F0F0F0,stroke:#666666,color:#000;
```
