How open-x-humanoid/tienkung-lab works
1. Repository shape
- The repo is split into two major codebases: legged_lab, which defines the TienKung robot task, simulation assets, sensors, and scripts; and rsl_rl, a vendored reinforcement-learning library used for PPO/AMP training and rollout management.
- The overall architecture is an IsaacLab-based robotics control stack, not a general application stack.
2. Core domain: TienKung locomotion
- The main task domain lives under legged_lab/envs/tienkung.
- It defines separate task configurations for walk, run, and sensor-augmented variants like walk_with_sensor and run_with_sensor.
- These configs feed into a common environment implementation in legged_lab/envs/tienkung/tienkung_env.py.
3. Environment framework
- legged_lab/envs/base provides reusable environment abstractions and configuration classes.
- This is the boundary where task-specific settings are layered on top of IsaacLab-compatible base environment behavior.
- The task registry in legged_lab/utils/task_registry.py likely connects CLI task names to the correct environment/config pair.
4. Robot assets and simulation representations
- Robot geometry and simulation assets are stored under legged_lab/assets/tienkung2_lite.
- The same robot is represented in multiple formats: STL meshes, URDF, MJCF, and USD.
- This is important architecturally because the repo supports both IsaacLab simulation and MuJoCo sim-to-sim evaluation, so asset conversion is part of the project structure.
5. Motion data pipeline
- Motion data is a first-class input to the system.
- The repo distinguishes two dataset roles under legged_lab/envs/tienkung/datasets:
- motion_visualization: used to replay or inspect motions.
- motion_amp_expert: used as expert demonstrations for AMP-style learning.
- The README shows a two-step conversion flow from GMR/SMPLX retargeting output into visualization data, then into AMP expert data.
6. Training and imitation-learning loop
- Training is driven by legged_lab/scripts/train.py.
- The learning setup combines reinforcement learning with AMP-style motion imitation; the README explicitly mentions AMP rewards plus periodic gait rewards.
- The actual RL algorithms and buffers come from the bundled rsl_rl package, especially amp_ppo.py, ppo.py, rollout_storage.py, and the AMP-related modules.
7. Policy execution and evaluation
- legged_lab/scripts/play.py runs a trained policy in IsaacLab.
- legged_lab/scripts/play_amp_animation.py is used to visualize motion playback and sensor-augmented playback, which acts as a debugging and dataset validation path.
- legged_lab/scripts/sim2sim.py evaluates exported policies in MuJoCo, forming a separate runtime boundary from IsaacLab.
8. Sensors and perceptual extensions
- The repo includes sensor packages under legged_lab/sensors, with camera and lidar support.
- Camera support includes device-specific configs such as D455 depth and Luxonis OAK-D, plus a tiled camera abstraction.
- These sensor modules are optional extensions used by the *_with_sensor tasks, indicating the architecture supports both proprioceptive-only and perceptive control paths.
9. Terrain and environment geometry
- Terrain-related logic lives under legged_lab/terrains.
- The ray-caster modules suggest environment sensing and/or terrain interaction are implemented with explicit geometric queries.
- This subsystem is likely shared by locomotion tasks that need obstacle awareness or local terrain perception.
10. Reward and control shaping
- legged_lab/mdp/rewards.py is the main place where task reward terms are defined.
- In practice, this is where locomotion objectives such as gait periodicity, tracking, stability, and AMP-related penalties/rewards are composed.
- This subsystem sits between raw simulation state and the learning algorithm.
11. Bundled RL library boundary
- rsl_rl is a self-contained reinforcement learning library included in the repo rather than consumed as a pure external dependency.
- It provides reusable algorithm implementations, actor-critic models, recurrent variants, AMP-specific training runner support, motion loaders, and logging utilities.
- This boundary matters because the project’s training stack is split between task-specific robotics code and generic RL infrastructure.
12. Utilities, CLI, and logging
- legged_lab/utils contains CLI argument handling, keyboard utilities, scene helpers, and environment setup helpers.
- The rsl_rl utilities include integrations for logging backends such as Weights & Biases and Neptune.
- These pieces are not the core control logic, but they connect training/evaluation workflows to user interaction and experiment tracking.
13. External dependencies and runtime constraints
- The architecture depends materially on IsaacSim 4.5.0 and IsaacLab 2.1.0, runs on Python 3.10/Linux, and uses MuJoCo for sim-to-sim validation.
- The README also references GMR for motion retargeting, which is upstream of the training pipeline.
- Exported_policy/walk.pt shows that trained policies are treated as deployable artifacts separate from the source code.
14. End-to-end flow
- The typical pipeline is: retarget motion data externally, convert it into repo-specific motion files, visualize and validate the motion, train an AMP+RL policy in IsaacLab, export the learned policy, then evaluate it either in IsaacLab playback or MuJoCo sim-to-sim.
- Real-robot deployment is outside this repo, but the README makes clear that the learned policy is intended to transfer to hardware via a separate deployment repository.
Main entry points: tienkung_env.py TienKung env · task_registry.py Task registry · runners Runner · tienkung2_lite Robot assets
Components 22
legged_lab
Task-specific robotics stack for TienKung locomotion, assets, sensors, and workflows.
- Robot assets · TienKung Lite robot represented across simulation formats and geometry sources.
legged_lab/assets/tienkung2_lite - USD model · Canonical IsaacLab robot asset and sensor/physics variants.
legged_lab/assets/tienkung2_lite/usd/tienkung2_lite.usd - Base env · Reusable environment abstractions and configuration layer for IsaacLab-compatible tasks.
legged_lab/envs/base - TienKung env · Common environment implementation shared by walk, run, and sensor variants.
legged_lab/envs/tienkung/tienkung_env.py - Walk cfg · Walking task configuration.
legged_lab/envs/tienkung/walk_cfg.py - Run cfg · Running task configuration.
legged_lab/envs/tienkung/run_cfg.py - Walk sensor · Walking task with camera/lidar sensor extensions.
legged_lab/envs/tienkung/walk_with_sensor_cfg.py - Run sensor · Running task with sensor extensions.
legged_lab/envs/tienkung/run_with_sensor_cfg.py - Task registry · Maps task names to the correct environment and configuration pair.
legged_lab/utils/task_registry.py - Rewards · Locomotion, tracking, stability, periodic gait, and AMP-related reward terms.
legged_lab/mdp/rewards.py - Motion data · Visualization and expert motion files used by playback and AMP training.
legged_lab/envs/tienkung/datasets - Sensors · Optional perceptual extensions for sensor-augmented control paths.
legged_lab/sensors - Terrains · Terrain generation and ray-caster geometry queries for environment interaction.
legged_lab/terrains - Train · Training entrypoint for AMP+RL policy learning.
legged_lab/scripts/train.py - Play · Runs a trained policy in IsaacLab for rollout and inspection.
legged_lab/scripts/play.py - AMP replay · Motion playback and sensor-augmented visualization path for debugging datasets and policies.
legged_lab/scripts/play_amp_animation.py - Sim2sim · Evaluates exported policies in MuJoCo as a separate runtime boundary.
legged_lab/scripts/sim2sim.py
rsl_rl
Bundled reinforcement-learning library used for PPO/AMP training and rollout management.
- Algorithms · Core PPO, AMP-PPO, and distillation implementations.
rsl_rl/rsl_rl/algorithms - Runner · Training loop orchestration for on-policy and AMP training.
rsl_rl/rsl_rl/runners - Storage · Rollout storage and replay buffers used by the training loop.
rsl_rl/rsl_rl/storage - Models · Actor-critic, discriminator, normalizer, RND, and student-teacher modules.
rsl_rl/rsl_rl/modules - RL utils · Motion loaders, logging integrations, and helper utilities for training and evaluation.
rsl_rl/rsl_rl/utils
Connections 27
- Task registry → Walk cfg: selects · Routes walk task name to its configuration.
- Task registry → Run cfg: selects · Routes run task name to its configuration.
- Task registry → Walk sensor: selects · Routes sensor walk task name to its configuration.
- Task registry → Run sensor: selects · Routes sensor run task name to its configuration.
- Walk cfg → TienKung env: configures · Task configuration feeds the shared environment implementation.
- Run cfg → TienKung env: configures · Task configuration feeds the shared environment implementation.
- Walk sensor → TienKung env: configures · Sensor task configuration feeds the shared environment implementation.
- Run sensor → TienKung env: configures · Sensor task configuration feeds the shared environment implementation.
- TienKung env → Base env: extends · Task environment builds on reusable base environment behavior.
- TienKung env → Rewards: uses · Environment computes locomotion and AMP-style reward shaping.
- TienKung env → Motion data: loads · Environment consumes motion visualization and expert AMP data.
- TienKung env → Sensors: optionally uses · Sensor-augmented variants attach perceptual inputs.
- TienKung env → Terrains: samples · Terrain and ray-caster logic provides environment geometry queries.
- TienKung env → Robot assets: instantiates · Environment uses the robot asset package.
- Robot assets → USD model: includes · Robot package contains the IsaacLab USD asset.
- Train → Task registry: resolves · Training entrypoint resolves the configured task.
- Train → Runner: starts · Training delegates loop execution to the RL runner.
- Runner → Algorithms: drives · Runner executes PPO/AMP optimization.
- Runner → Storage: stores · Runner records rollouts and replay data.
- Algorithms → Models: uses · Algorithms depend on policy, discriminator, and auxiliary modules.
- Runner → RL utils: logs · Runner uses motion loaders and experiment logging utilities.
- Play → Task registry: loads · Play entrypoint resolves the policy task.
- Play → TienKung env: runs · Runs trained policy inside the IsaacLab environment.
- AMP replay → Motion data: replays · Visualization entrypoint inspects motion datasets.
- AMP replay → Sensors: visualizes · Also supports sensor-augmented playback.
- Sim2sim → Robot assets: evaluates · MuJoCo sim-to-sim path uses exported robot assets and policy artifacts.
- Sim2sim → RL utils: loads · Uses RL utilities and exported-policy handling for evaluation.