How unitreerobotics/unitree_sim_isaaclab works
Project type and purpose
• This is a full-stack simulation framework built on NVIDIA Isaac Lab / Isaac Sim.
• It simulates Unitree robots (G1 / H1-2 variants) performing manipulation and mobile tasks via Gym-style RL environments.
• It uses DDS for robot command/feedback compatibility with real-robot code, ZMQ for image streaming, and a modular, plugin-based task system.
File structure
Top-level modules and their roles:
• sim_main.py – entry point: parses CLI args, initializes Isaac Lab, selects task/env, launches main loop.
• action_provider/ – abstractions for sourcing robot actions: file replay, DDS messages, or policy outputs.
• dds/ – implements Fast-DDS publishers/subscribers matching Unitree SDK2 topics for G1 robot body, gripper, dex3 hand.
• image_server/ – ZMQ-based image publisher for camera streams.
• layeredcontrol/ – low-level controller that takes “actions” and applies them to the simulated robot joints.
• robots/ – static robot definitions (URDF converters, presets for Unitree variants).
• tasks/ – gym-registered task modules split into:
– common_* (configs, scenes, observations, terminations, rewards)
– g1_tasks, h1-2_tasks (per-robot scenario directories with mdp subpackages)
– utils (Hydra config parsing, importer hooks)
• tools/ – standalone scripts for USD asset manipulation, data conversion, augmentation, reward computation.
• doc/, Dockerfile, requirements.txt, fetch_assets.sh – install & deployment support.
README for additional insights
• Emphasizes: IsaacSim 4.5/5.0 requirement, hardware recommendations.
• DDS topics mirror real-robot protocols.
• Tasks are registered via gym.register in each tasks/<robot>_tasks/__init__.py.
• Data generation and replay modes use --replay, --generate_data flags.
• High cohesion, low coupling: communication, control, scene-building, and tasks are clearly separated.
Main components
a. sim_main (CLI & orchestrator)
b. Isaac Lab/Isaac Sim runtime (scene graph, physics, USD stage)
c. Task Manager (gym-based RL Env Manager)
d. Tasks module (common_scene, common_observations, common_termination, robot-specific MDP configs)
e. Action Provider (DDS client / file reader / policy output)
f. Layered Control (applies joint commands into IsaacSim)
g. DDS communication layer (Fast-DDS pub/sub)
h. Image Server (ZMQ pub/sub for camera feeds)
Relationships & data flows
1) sim_main → initializes Isaac Sim (loads USD, cameras, robot)
2) sim_main → registers & instantiates a Gym Env (tasks modules)
3) In each simulation step:
• Task env → requests observations (common_observations) → physics state & camera images
• Observations → returned to action_provider or external RL policy
• Action_provider → selects source: DDS messages (live teleop), replay file, or policy → outputs action vector
• Layeredcontrol → sends action vector into IsaacSim joint controllers
• DDS module → if enabled, publishes/receives messages to mimic real-robot comms
• Image_server → continuously pulls camera frames from IsaacSim & pushes over ZMQ
4) Task env → computes rewards & terminations (common_rewards, common_termination)
Architectural patterns & design principles
• Modular separation: Communication (DDS, ZMQ) vs. Control vs. Scene config vs. Tasks
• Plugin-based tasks: new tasks register via gym.register; no core changes needed
• Layered control: high-level RL vs. low-level joint actuation
• Reuse of common components: common_scene, common_observations, common_termination