# unitreerobotics/unitree_sim_isaaclab architecture

> Architecture diagram of the GitHub repository unitreerobotics/unitree_sim_isaaclab, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-10-05). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/unitreerobotics/unitree_sim_isaaclab
- Repository: https://github.com/unitreerobotics/unitree_sim_isaaclab

## Overview

##### Project type and purpose
- This is a full-stack simulation framework built on NVIDIA Isaac Lab / Isaac Sim.
- It simulates Unitree robots (G1 / H1-2 variants) performing manipulation and mobile tasks via Gym-style RL environments.
- It uses DDS for robot command/feedback compatibility with real-robot code, ZMQ for image streaming, and a modular, plugin-based task system.

##### File structure
Top-level modules and their roles:
- sim_main.py – entry point: parses CLI args, initializes Isaac Lab, selects task/env, launches main loop.
- action_provider/ – abstractions for sourcing robot actions: file replay, DDS messages, or policy outputs.
- dds/ – implements Fast-DDS publishers/subscribers matching Unitree SDK2 topics for G1 robot body, gripper, dex3 hand.
- image_server/ – ZMQ-based image publisher for camera streams.
- layeredcontrol/ – low-level controller that takes “actions” and applies them to the simulated robot joints.
- robots/ – static robot definitions (URDF converters, presets for Unitree variants).
- tasks/ – gym-registered task modules split into:
– common_* (configs, scenes, observations, terminations, rewards)
– g1_tasks, h1-2_tasks (per-robot scenario directories with mdp subpackages)
– utils (Hydra config parsing, importer hooks)
- tools/ – standalone scripts for USD asset manipulation, data conversion, augmentation, reward computation.
- doc/, Dockerfile, requirements.txt, fetch_assets.sh – install & deployment support.

##### README for additional insights
- Emphasizes: IsaacSim 4.5/5.0 requirement, hardware recommendations.
- DDS topics mirror real-robot protocols.
- Tasks are registered via gym.register in each tasks/<robot>_tasks/__init__.py.
- Data generation and replay modes use --replay, --generate_data flags.
- High cohesion, low coupling: communication, control, scene-building, and tasks are clearly separated.

##### Main components
a. sim_main (CLI & orchestrator)
b. Isaac Lab/Isaac Sim runtime (scene graph, physics, USD stage)
c. Task Manager (gym-based RL Env Manager)
d. Tasks module (common_scene, common_observations, common_termination, robot-specific MDP configs)
e. Action Provider (DDS client / file reader / policy output)
f. Layered Control (applies joint commands into IsaacSim)
g. DDS communication layer (Fast-DDS pub/sub)
h. Image Server (ZMQ pub/sub for camera feeds)

##### Relationships & data flows
1) sim_main → initializes Isaac Sim (loads USD, cameras, robot)
2) sim_main → registers & instantiates a Gym Env (tasks modules)
3) In each simulation step:
- Task env → requests observations (common_observations) → physics state & camera images
- Observations → returned to action_provider or external RL policy
- Action_provider → selects source: DDS messages (live teleop), replay file, or policy → outputs action vector
- Layeredcontrol → sends action vector into IsaacSim joint controllers
- DDS module → if enabled, publishes/receives messages to mimic real-robot comms
- Image_server → continuously pulls camera frames from IsaacSim & pushes over ZMQ
4) Task env → computes rewards & terminations (common_rewards, common_termination)

##### Architectural patterns & design principles
- Modular separation: Communication (DDS, ZMQ) vs. Control vs. Scene config vs. Tasks
- Plugin-based tasks: new tasks register via gym.register; no core changes needed
- Layered control: high-level RL vs. low-level joint actuation
- Reuse of common components: common_scene, common_observations, common_termination

## Diagram (Mermaid)

```mermaid
flowchart TD
    %% Orchestrator
    subgraph "Orchestrator"
        SM["sim_main.py\n(Python CLI)"]:::orchestrator
    end
    click SM "https://github.com/unitreerobotics/unitree_sim_isaaclab/blob/main/sim_main.py"

    %% Simulation Core
    subgraph "Simulation Core"
        IS["IsaacSim Runtime\n(USD, Physics)"]:::simulation
        LC["Layered Control\n(Python)"]:::simulation
    end
    click LC "https://github.com/unitreerobotics/unitree_sim_isaaclab/blob/main/layeredcontrol/robot_control_system.py"

    %% Communication Layer
    subgraph "Communication Layer"
        DDS["DDS Comm\n(Fast-DDS Pub/Sub)"]:::communication
        IZ["Image Server\n(ZMQ)"]:::communication
    end
    click DDS "https://github.com/unitreerobotics/unitree_sim_isaaclab/tree/main/dds/"
    click IZ "https://github.com/unitreerobotics/unitree_sim_isaaclab/blob/main/image_server/image_server.py"
    click IZ "https://github.com/unitreerobotics/unitree_sim_isaaclab/blob/main/image_server/shared_memory_utils.py"

    %% Task Layer
    subgraph "Task Layer"
        TM["Task Manager\n(Gym Env)"]:::task
        CS["Common Scene"]:::task
        CO["Common Observations"]:::task
        CR["Common Rewards"]:::task
        CT["Common Termination"]:::task
        CC["Common Config"]:::task
        GP["G1 Tasks MDP"]:::task
        HP["H1-2 Tasks MDP"]:::task
    end
    click TM "https://github.com/unitreerobotics/unitree_sim_isaaclab/blob/main/tasks/__init__.py"
    click TM "https://github.com/unitreerobotics/unitree_sim_isaaclab/blob/main/tasks/utils/importer.py"
    click CS "https://github.com/unitreerobotics/unitree_sim_isaaclab/tree/main/tasks/common_scene/"
    click CO "https://github.com/unitreerobotics/unitree_sim_isaaclab/tree/main/tasks/common_observations/"
    click CR "https://github.com/unitreerobotics/unitree_sim_isaaclab/tree/main/tasks/common_rewards/"
    click CT "https://github.com/unitreerobotics/unitree_sim_isaaclab/tree/main/tasks/common_termination/"
    click CC "https://github.com/unitreerobotics/unitree_sim_isaaclab/tree/main/tasks/common_config/"
    click GP "https://github.com/unitreerobotics/unitree_sim_isaaclab/tree/main/tasks/g1_tasks/"
    click HP "https://github.com/unitreerobotics/unitree_sim_isaaclab/tree/main/tasks/h1-2_tasks/"

    %% Action Sources
    subgraph "Action Sources"
        AP["Action Provider\n(Python)"]:::action
        APF["Replay Files"]:::action
        POL["Policy Outputs"]:::action
    end
    click AP "https://github.com/unitreerobotics/unitree_sim_isaaclab/tree/main/action_provider/"

    %% Utilities
    subgraph "Utilities"
        RB["Robots Definitions"]:::utils
        TL["Tools & Utilities"]:::utils
    end
    click RB "https://github.com/unitreerobotics/unitree_sim_isaaclab/blob/main/robots/unitree.py"
    click TL "https://github.com/unitreerobotics/unitree_sim_isaaclab/tree/main/tools/"

    %% External
    EX["External Clients\n(ROS / Teleop)"]:::external

    %% Connections
    SM -->|"initialize"| IS
    SM -->|"register tasks"| TM

    TM -->|"load scenes"| CS
    TM -->|"observe"| CO
    TM -->|"compute reward"| CR
    TM -->|"check termination"| CT

    TM -->|"step env"| IS
    IS -->|"state feedback"| TM

    TM -->|"request action"| AP
    APF -->|"file replay"| AP
    POL -->|"policy output"| AP

    AP -->|"action vector"| LC
    LC -->|"apply commands"| IS

    DDS <-->|"robot topics"| AP
    LC -->|"mirror state"| DDS

    IS -->|"camera frames"| IZ
    IZ -->|"ZMQ stream"| EX

    RB -->|"robot presets"| IS

    %% Styles
    classDef orchestrator fill:#EEEEEE,stroke:#333,stroke-width:1px;
    classDef simulation fill:#DFF0D8,stroke:#333,stroke-width:1px;
    classDef communication fill:#D9EDF7,stroke:#333,stroke-width:1px;
    classDef task fill:#FCF8E3,stroke:#333,stroke-width:1px;
    classDef action fill:#F2DEDE,stroke:#333,stroke-width:1px;
    classDef utils fill:#F5F5F5,stroke:#333,stroke-width:1px;
    classDef external fill:#E8E8E8,stroke-dasharray: 5 5,stroke:#333,stroke-width:1px;
```
