How unitreerobotics/unitree_rl_gym works
Based on our analysis of the repository’s file tree and README, this project is a reinforcement learning framework focused on motion control for Unitree robots. The repository supports multiple deployment scenarios—training in simulation (using Isaac Gym), transferring to another simulator (Mujoco with Sim2Sim), and finally deploying on real hardware (Sim2Real). The architecture is modular, with clear separations between training, simulation, deployment, and resource definitions.
Project Type & Purpose
• The project is a reinforcement learning (RL) system that trains robotic policies using simulation and then deploys those policies to both simulated environments (Mujoco) and physical Unitree robots.
• It supports multiple robot models (e.g., Go2, G1, H1, H1_2) and demonstrates an end-to-end workflow: Train → Play → Sim2Sim → Sim2Real.
Main Components
• Training Environment (legged_gym):
• Environment definitions (in legged_gym/envs): Includes base configurations and robot-specific configurations.
• Training scripts (legged_gym/scripts/train.py): Implements reinforcement learning policy optimization.
• Utility modules (legged_gym/utils): Helpers, logging, math utilities, and task registry.
• Play Module:
• Play script (legged_gym/scripts/play.py) that loads and visualizes trained models.
• Policy export mechanism that saves actor networks in designated folders.
• Deployment for Simulation (Sim2Sim):
• Deployment module under deploy/deploy_mujoco, with YAML configuration files (configs/) and a Python deployment script (deploy/deploy_mujoco.py).
• Deployment for Physical Robots (Sim2Real):
• Deployment module under deploy/deploy_real: Contains specific scripts and helper modules (e.g., command_helper.py, remote_controller.py) for interfacing with real hardware.
• Configuration files (deploy/deploy_real/configs/) to match specific robot settings.
• Pre-Trained Models:
• Located under deploy/pre_train: Contains default motion policies that the deployments (Sim2Sim/Sim2Real) can load, with the option to replace them with custom-trained models.
• Resource Files:
• The resources folder contains URDF robot descriptions, meshes, and additional simulation assets required to accurately represent the robots in the simulation.
Component Relationships and Data Flow
• The Play module then accesses the training logs and saved policy files and visualizes the policy’s performance.
• Once a policy is verified, it can be exported (as detailed in the README) and loaded by the deployment modules.
• In Sim2Sim (Mujoco deployment), the deploy/deploy_mujoco module reads a YAML configuration file to launch a simulation instance in Mujoco, applying the exported policy.
• In Sim2Real, the deploy/deploy_real module uses additional components (like a remote controller) to send commands from the policy to a physical robot via a specified network interface.
• Both deployment scenarios depend on the resource files (robot models and meshes) to accurately represent the robot’s structure.
Architectural Patterns/Design Principles
• Modularity: Each major functionality (training, simulation, deployment) is grouped into distinct directories.
• Configuration-Driven: YAML files in deploy/deploy_mujoco and deploy/deploy_real enable easy switching between robot models and parameter tuning.
• Separation of Concerns: Training environment, policy visualization, simulated deployment, and physical deployment are separated, allowing independent development and testing.
• Extensibility: The structure permits new robots to be added by extending environment configurations and resource files.