How czy412/lidar_hexapod_rl works
1. This repository is an Isaac Gym–based robotics stack for simulating and training legged robots and one manipulator. It is centered on environment definitions, robot assets, terrain generation, and a separate reinforcement-learning library used to train policies against those environments.
2. The repo is split into two major subsystems: legged_gym and rsl_rl. legged_gym contains all robot/environment/task logic plus simulation assets and runnable scripts. rsl_rl contains the on-policy RL algorithms, rollout storage, neural network modules, and training runners that consume environment observations and produce actions.
3. legged_gym/legged_gym/envs/base is the common simulation foundation. It defines generic robot/task abstractions, reward mixins, pose-adaptation base classes, and robot config schemas. Higher-level environments inherit from these base classes rather than duplicating simulator setup, observation assembly, reward computation, and reset logic.
4. Robot-specific environments are organized by platform: a1, anymal_b, anymal_c, go2, cassie, cyberdog2, elspider_air, and franka. Each robot has a base environment class plus one or more config modules, with variants for flat ground, rough/mixed terrain, navigation, pose adaptation, batch rollout, and gradient-sampling use cases. The architecture is config-driven: the environment class is usually shared while behavior changes via task-specific config files.
5. batch_rollout is an important architectural extension. Instead of only running single-task RL episodes, these environments support “main-rollout” style sampling for planning, navigation, trajectory-gradient estimation, and teacher/student or MPC-like workflows. There are parallel branches such as nav, plan-grad sampling, traj-grad sampling, and dial-MPC-oriented configs, which suggests the environment can act both as an RL simulator and as a sampling engine for downstream planners or optimizers.
6. Terrain and world modeling is a major subsystem. legged_gym/utils/terrain.py, terrain_obj.py, and terrain_confine.py support procedurally generated terrain, OBJ-based terrains, and confined terrains. The resources folder includes terrain meshes, and the README highlights confined terrain generation as a first-class feature. This terrain layer feeds directly into environment initialization and influences observation/reward/reset behavior.
7. Perception and contact sensing are treated as separate simulation modalities. The base envs include legged_robot_depthcam.py and legged_robot_raycast.py, while elspider_air adds elspider_lidar.py. Utility modules like depth_camera.py, ray_caster.py, and mesh_sdf.py indicate the repo uses Nvidia Warp/SDF and raycasting for geometric sensing. These sensors expand observations beyond proprioception and are likely used by navigation and terrain-estimation tasks.
8. The training loop flows from legged_gym/scripts/train.py and play.py into rsl_rl/runners/on_policy_runner.py and the PPO/distillation implementations. Environments expose vectorized observations, rewards, resets, and privileged state; rsl_rl then collects rollouts, normalizes inputs, computes policy/value losses, and updates actor-critic networks. The environment-to-policy boundary is clean: simulation lives in legged_gym, optimization lives in rsl_rl.
9. rsl_rl is structured as a reusable learning engine. Key parts include algorithms/ppo.py and distillation.py, network modules for actor-critic, recurrent variants, student-teacher models, depth backbones, and terrain estimators, plus storage and vector-env interfaces. This separation makes it possible to swap policy architectures without changing the simulator code.
10. There is explicit support for teacher/student and auxiliary estimation tasks. Modules such as student_teacher.py, terrain_estimator.py, depth_backbone.py, and rnd.py indicate the policy can be trained with privileged information, depth perception, or auxiliary prediction heads. The environment side mirrors this with task variants that likely provide extra observations or labels during training.
11. The repository includes broad asset management under legged_gym/resources. URDFs, meshes, and licenses are separated by robot, with actuator nets and robot-specific model files bundled for simulation. This asset layer is the physical boundary between authored robot descriptions and the runtime simulator.
12. Operational tooling is split between scripts, tests, and utilities. scripts/terrain_est_train.py and terrain_est_play.py show a dedicated terrain-estimation training path in addition to standard RL training. The tests tree covers ray casting, mesh SDFs, batch rollout behavior, environment stepping, and specific robot checks, which helps validate the simulator and geometry stack independently of policy training.
13. External integrations affect architecture but are not the core of it: Isaac Gym is the physics/simulation runtime; Nvidia Warp appears in terrain/sensing features; Weights & Biases and Neptune hooks are present in rsl_rl/utils, indicating optional experiment tracking. The repository is designed to be embedded as a submodule in PegasusFlow, so its primary role is a reusable simulation-and-training backend rather than a standalone application.
Main entry points: base Base env · batch_rollout Batch rollout · on_policy_runner.py Runner · tests Tests
Components 18
legged_gym
Isaac Gym robotics simulation, assets, tasks, sensing, and runnable scripts.
- Base env · Common task, robot, reward, and config abstractions shared by all environments.
legged_gym/legged_gym/envs/base - Terrain · Procedural, OBJ, and confined terrain generation feeding environment initialization.
legged_gym/legged_gym/utils/terrain.py - Sensing · Depth, raycast, lidar, and mesh-SDF utilities used to extend observations.
legged_gym/legged_gym/utils/depth_camera.py - Batch rollout · Main-rollout and gradient-sampling environment variants for planning, navigation, and MPC-like workflows.
legged_gym/legged_gym/envs/batch_rollout - ANYmal C · ANYmal C locomotion, pose adaptation, navigation, and batch-rollout variants.
legged_gym/legged_gym/envs/anymal_c - Go2 · Go2 locomotion and batch-rollout variants driven by shared configs.
legged_gym/legged_gym/envs/go2 - Elspider Air · Elspider Air locomotion, lidar, navigation, and gradient-sampling variants.
legged_gym/legged_gym/envs/elspider_air - Other robots · Additional robot-specific environments for A1, ANYmal B, Cassie, Cyberdog2, and Franka.
legged_gym/legged_gym/envs - Assets · URDFs, meshes, licenses, actuator nets, and terrain meshes consumed by simulation.
legged_gym/resources - Scripts · Training, playback, and terrain-estimation entrypoints that launch environments and learning.
legged_gym/legged_gym/scripts - Utilities · Shared helpers for logging, planning, visualization, math, registry, and replay.
legged_gym/legged_gym/utils - Tests · Geometry, ray casting, rollout, environment, and robot-specific checks.
legged_gym/legged_gym/tests
rsl_rl
Reusable on-policy reinforcement learning library used to train policies on legged_gym environments.
- Runner · On-policy orchestration that collects rollouts and drives optimization.
rsl_rl/rsl_rl/runners/on_policy_runner.py - Algorithms · PPO and distillation implementations that update policies from collected experience.
rsl_rl/rsl_rl/algorithms - Policy nets · Actor-critic, recurrent, student-teacher, depth, terrain, and RND modules.
rsl_rl/rsl_rl/modules - Storage · Rollout buffers and sequence memory used by on-policy learning.
rsl_rl/rsl_rl/storage - Vec env · Vectorized environment interface used to connect simulators to the learner.
rsl_rl/rsl_rl/env/vec_env.py - Tracking · Optional experiment tracking helpers for Weights & Biases and Neptune.
rsl_rl/rsl_rl/utils
Connections 21
- Base env → ANYmal C: inherits · Robot environments extend the shared base simulation and reward abstractions.
- Base env → Go2: inherits · Go2 environments reuse the common legged robot/task foundation.
- Base env → Elspider Air: inherits · Elspider Air environments build on the base robot/task classes.
- Base env → Other robots: inherits · A1, ANYmal B, Cassie, Cyberdog2, and Franka share the common base patterns.
- Terrain → Base env: configures · Terrain generation shapes environment initialization, resets, and reward structure.
- Sensing → Base env: extends obs · Depth, raycast, lidar, and SDF modalities extend base observations.
- Batch rollout → ANYmal C: specializes · Batch-rollout and gradient-sampling branches are implemented per robot.
- Batch rollout → Go2: specializes · Go2 batch-rollout variants support planning and trajectory-gradient workflows.
- Batch rollout → Elspider Air: specializes · Elspider Air exposes batch rollout, nav, dial-MPC, and sampling configs.
- Assets → Base env: loads · URDFs, meshes, and licenses provide the physical robot and terrain models.
- Scripts → Runner: launches · Training and play scripts hand control to the RL runner.
- Vec env → Base env: consumes · The learner interacts with vectorized observations, rewards, and resets.
- Runner → Storage: collects · Rollouts are stored before policy updates.
- Runner → Algorithms: optimizes with · The runner invokes PPO or distillation updates.
- Algorithms → Policy nets: trains · Algorithms update actor-critic and auxiliary network modules.
- Policy nets → Storage: reads batches · Policy modules consume rollout and memory data during training.
- Policy nets → Sensing: uses inputs · Depth and terrain-estimation modules align with simulator sensing outputs.
- Tracking → Runner: tracks · Optional logging and experiment tracking wrap training runs.
- Tests → Sensing: validates · Ray casting and mesh-SDF tests exercise the sensing stack.
- Tests → Batch rollout: validates · Batch rollout tests cover planning and sampling behavior.
- Tests → Base env: validates · Environment and step-time tests cover the common simulator foundation.