How hybridrobotics/whole_body_tracking works
Project Type & Purpose
• This is neither a traditional web app nor a pure library—it’s an end-to-end RL motion-tracking pipeline.
• Core purpose: preprocess human motion data, host motions in WandB, run physics-based simulation (IsaacSim/IsaacLab), train/test RL policies (PPO via RSL-RL), and log artifacts.
High-Level Components
A. Data Ingestion & Processing
• Input: raw .csv motion datasets (e.g. LAFAN1).
• scripts/csv_to_npz.py → forward-kinematics npz files.
• Upload npz artifacts to WandB registry.
B. Simulation Environment
• IsaacSim v4.5.0 + IsaacLab v2.1.0 (external service).
• Uses robot descriptions (source/.../assets) and MDP definitions.
C. Reinforcement Learning Core
• Library package (source/whole_body_tracking/whole_body_tracking):
• tasks/tracking/mdp: commands, rewards, observations, terminations, events
• tasks/tracking/tracking_env_cfg.py (env config)
• config/g1 & config/humanoid (agent hyperparams)
• robots/ (actuator models, joint parameters)
• utils/ exporter & my_on_policy_runner (training loop)
D. Orchestration Scripts
• scripts/rsl_rl/train.py (launches training job, logger=wandb)
• scripts/rsl_rl/play.py (policy evaluation/replay)
• scripts/replay_npz.py (quick sim visualization)
• scripts/upload_npz.py (alternative upload)
E. External Services & Artifacts
• WandB registry for storing/loading motions & run logs
• GCS (robot description tarball)
• Conda/Python environment (packaged via pyproject.toml & setup.py)
Interactions & Data Flow
• Raw CSV motion → (csv_to_npz) → NPZ artifact → WandB registry
• train.py:
• Pull NPZ from WandB → initialize simulation MDP → run PPO (my_on_policy_runner) → log metrics & model artifacts back to WandB
• play.py / replay_npz.py:
• Pull NPZ or trained policy from WandB → launch IsaacSim → render trajectories
• Core library modules (mdp, config, robots) are loaded at runtime by scripts.
Architectural Patterns & Principles
• Modular separation:
• Data preprocessing (scripts) vs. core algorithm (library) vs. orchestration (train/play scripts).
• Plugin/config pattern:
• TOML configs under tasks/tracking/config drive env & agent parameters.
• Dependency injection:
• CLI args select tasks (e.g. “Tracking-Flat-G1-v0”), wandb paths, headless mode.