How telehuman/pbhc works
Project Type and Purpose
• This is a research-grade, full-stack robotics and RL framework (“KungfuBot”) for physics-based humanoid control.
• It integrates a multi-stage motion-processing pipeline, an RL training/deployment engine, and visualization/tools modules.
High-Level Architecture
The system can be viewed as a linear pipeline of stages with cross-cutting configuration and external dependencies:
┌───────────────┐ ┌─────────────────┐ ┌────────────────────┐ ┌───────────────────┐ ┌───────────────────┐
│ Motion Source │ → │ Retargeting │ → │ Motion Processing │ → │ RL Training │ → │ Deployment & Eval │
└───────────────┘ └─────────────────┘ └────────────────────┘ └───────────────────┘ └───────────────────┘
(motion_source/) (smpl_retarget/) (robot_motion_process/, (humanoidverse/) (humanoidverse/)
smpl_vis/)
Main Components and Their Interactions
a. Motion Collection (“motion_source/”)
• Input: raw human motion (video, LAFAN, AMASS, custom).
• Output: unified SMPL-format files (*.pkl).
• Key modules: convert_lafan_pkl.py, demo.py, rotation-utils, batch utilities.
b. Retargeting (“smpl_retarget/”)
• Pipelines: Mink-based (mink_retarget/) and PHC-based (phc_retarget/).
• Input: SMPL motions.
• Output: robot-specific motion trajectories.
• Tools: motion_filter (denoise/normalize), poselib for skeleton abstractions.
c. Motion Processing & Visualization
• “robot_motion_process/”: interpolation, trajectory analysis, MJViewer scripts.
• “smpl_vis/”: SMPL mesh rendering, Blender import, custom renderer tools.
• Purpose: QC and analysis before RL training.
d. RL Training Framework (“humanoidverse/”)
• Core: Hydra-driven config, ASAP + RSL_RL libraries, modular agents, callbacks.
• Agents: base_algo, mh_ppo, ppo.
• Simulators: IsaacGym, MuJoCo, Genesis, IsaacSim.
• Envs: motion_tracking, legged_base, base_task.
• Deploy: single vs multiple robot setups, external modules for real-world.
e. Examples & Data (“example/”)
• Sample SMPL motions, pretrained policy checkpoints (PyTorch + ONNX).
f. Configuration & Utilities
• “humanoidverse/config/”: Hydra config hierarchy (algo, env, sim, robot, terrain, domain randomization, logging, deploy).
• “humanoidverse/utils/”: logging, math, config helpers, motion libraries, real-robot helpers.
External Dependencies & Data Stores
• External libraries: ASAP, RSL_RL, Maskedmimic (Mink), PHC, GVHMR, IPMAN.
• Simulators: NVIDIA IsaacGym, MuJoCo, IsaacSim, Genesis.
• Data: SMPL datasets, Unitree G1 URDF & meshes (under description/robots/g1), pretrained models (.pt, .onnx).