# nvlabs/gr00t-wholebodycontrol architecture

> Architecture diagram of the GitHub repository nvlabs/gr00t-wholebodycontrol, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-09-29). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/nvlabs/gr00t-wholebodycontrol
- Repository: https://github.com/nvlabs/gr00t-wholebodycontrol

## Overview

This repository brings together humanoid whole-body control, teleoperation, simulation and deployment, plus the SONIC and MotionBricks projects described in the README. A principal workflow is operator or streamed input through teleoperation/control components toward a simulated or deployed robot, with data-export tooling supporting collection. The tree and excerpts identify these subsystems, but do not establish all their runtime wiring; edges are limited to documented behavior and sampled calls. SONIC training and MotionBricks capabilities are README-level context, not inspected implementation paths.

Main entry points: [`decoupled_wbc/control/main/teleop/run_g1_control_loop.py`](https://github.com/nvlabs/gr00t-wholebodycontrol/blob/main/decoupled_wbc/control/main/teleop/run_g1_control_loop.py) (Control and simulation loops), [`gear_sonic_deploy/src/g1`](https://github.com/nvlabs/gr00t-wholebodycontrol/tree/main/gear_sonic_deploy/src/g1) (SONIC robot deployment), [`decoupled_wbc/control/teleop/gui/main.py`](https://github.com/nvlabs/gr00t-wholebodycontrol/blob/main/decoupled_wbc/control/teleop/gui/main.py) (Teleop devices and UI), [`decoupled_wbc/control/teleop/teleop_streamer.py`](https://github.com/nvlabs/gr00t-wholebodycontrol/blob/main/decoupled_wbc/control/teleop/teleop_streamer.py) (Teleop streamers)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_teleop["Teleoperation"]
  node_teleop_ui["Teleop devices and UI<br/>[main.py]"]
  node_teleop_stream["Teleop streamers<br/>[teleop_streamer.py]"]
  node_retarget["Retargeting and IK"]
end

subgraph group_decoupled["Decoupled WBC"]
  node_decoupled_policy["G1 WBC policies"]
  node_control_loops["Control and simulation loops"]
  node_g1_env["G1 environment<br/>[g1_env.py]"]
  node_sim_factory["Simulator selection"]
  node_simulators["G1 simulators<br/>[robocasa_sim.py]"]
  node_data_export["Data export<br/>[exporter.py]"]
end

subgraph group_sonic["GEAR-SONIC"]
  node_sonic_env["SONIC environment wrapper"]
  node_sonic_deploy["SONIC TensorRT inference"]
  node_zmq_input["ZMQ motion input<br/>[zmq_manager.hpp]"]
  node_gamepad_input["Gamepad input"]
  node_sonic_robot["SONIC robot deployment"]
end

subgraph group_motion["MotionBricks"]
  node_motion_controller["Interactive motion controllers<br/>[controllers.py]"]
end

node_operator(("Operator"))
node_real_robot(("G1 robot"))

node_operator -->|"operates"| node_teleop_ui
node_teleop_ui -->|"starts streams"| node_teleop_stream
node_teleop_stream -->|"streams poses"| node_retarget
node_operator -->|"selects control"| node_control_loops
node_control_loops -->|"runs policy"| node_decoupled_policy
node_control_loops -->|"steps environment"| node_g1_env
node_g1_env -->|"selects simulator"| node_sim_factory
node_sim_factory -->|"creates simulator"| node_simulators
node_control_loops -.->|"controls robot"| node_real_robot
node_control_loops -.->|"exports data"| node_data_export
node_sonic_env -.->|"decodes actions"| node_sonic_deploy
node_operator -->|"operates"| node_gamepad_input
node_operator -->|"streams commands"| node_zmq_input
node_gamepad_input -->|"selects input"| node_sonic_robot
node_zmq_input -->|"provides motion"| node_sonic_robot
node_sonic_deploy -->|"infers control"| node_sonic_robot
node_operator -->|"sets motion"| node_motion_controller

click node_teleop_ui "https://github.com/nvlabs/gr00t-wholebodycontrol/blob/main/decoupled_wbc/control/teleop/gui/main.py"
click node_teleop_stream "https://github.com/nvlabs/gr00t-wholebodycontrol/blob/main/decoupled_wbc/control/teleop/teleop_streamer.py"
click node_retarget "https://github.com/nvlabs/gr00t-wholebodycontrol/blob/main/decoupled_wbc/control/teleop/teleop_retargeting_ik.py"
click node_decoupled_policy "https://github.com/nvlabs/gr00t-wholebodycontrol/blob/main/decoupled_wbc/control/policy/g1_decoupled_whole_body_policy.py"
click node_control_loops "https://github.com/nvlabs/gr00t-wholebodycontrol/blob/main/decoupled_wbc/control/main/teleop/run_g1_control_loop.py"
click node_g1_env "https://github.com/nvlabs/gr00t-wholebodycontrol/blob/main/decoupled_wbc/control/envs/g1/g1_env.py"
click node_sim_factory "https://github.com/nvlabs/gr00t-wholebodycontrol/blob/main/decoupled_wbc/control/envs/g1/sim/simulator_factory.py"
click node_simulators "https://github.com/nvlabs/gr00t-wholebodycontrol/blob/main/decoupled_wbc/control/envs/g1/sim/robocasa_sim.py"
click node_data_export "https://github.com/nvlabs/gr00t-wholebodycontrol/blob/main/decoupled_wbc/data/exporter.py"
click node_sonic_env "https://github.com/nvlabs/gr00t-wholebodycontrol/blob/main/gear_sonic/envs/wrapper/manager_env_wrapper.py"
click node_sonic_deploy "https://github.com/nvlabs/gr00t-wholebodycontrol/blob/main/gear_sonic_deploy/src/TRTInference/InferenceEngine.cpp"
click node_zmq_input "https://github.com/nvlabs/gr00t-wholebodycontrol/blob/main/gear_sonic_deploy/src/g1/g1_deploy_onnx_ref/include/input_interface/zmq_manager.hpp"
click node_gamepad_input "https://github.com/nvlabs/gr00t-wholebodycontrol/blob/main/gear_sonic_deploy/src/g1/g1_deploy_onnx_ref/include/input_interface/gamepad_manager.hpp"
click node_sonic_robot "https://github.com/nvlabs/gr00t-wholebodycontrol/tree/main/gear_sonic_deploy/src/g1"
click node_motion_controller "https://github.com/nvlabs/gr00t-wholebodycontrol/blob/main/motionbricks/motionbricks/motion_backbone/demo/controllers.py"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_teleop_ui,node_teleop_stream,node_retarget toneBlue
class node_decoupled_policy,node_control_loops,node_g1_env,node_sim_factory,node_simulators,node_data_export toneAmber
class node_sonic_env,node_sonic_deploy,node_zmq_input,node_gamepad_input,node_sonic_robot toneMint
class node_motion_controller toneRose
class node_operator,node_real_robot toneIndigo
```

## Components

### Teleoperation

- [Teleop devices and UI](https://github.com/nvlabs/gr00t-wholebodycontrol/blob/main/decoupled_wbc/control/teleop/gui/main.py): `decoupled_wbc/control/teleop/gui/main.py`
- [Teleop streamers](https://github.com/nvlabs/gr00t-wholebodycontrol/blob/main/decoupled_wbc/control/teleop/teleop_streamer.py): `decoupled_wbc/control/teleop/teleop_streamer.py`
- [Retargeting and IK](https://github.com/nvlabs/gr00t-wholebodycontrol/blob/main/decoupled_wbc/control/teleop/teleop_retargeting_ik.py): `decoupled_wbc/control/teleop/teleop_retargeting_ik.py`

### Decoupled WBC

- [G1 WBC policies](https://github.com/nvlabs/gr00t-wholebodycontrol/blob/main/decoupled_wbc/control/policy/g1_decoupled_whole_body_policy.py): `decoupled_wbc/control/policy/g1_decoupled_whole_body_policy.py`
- [Control and simulation loops](https://github.com/nvlabs/gr00t-wholebodycontrol/blob/main/decoupled_wbc/control/main/teleop/run_g1_control_loop.py): `decoupled_wbc/control/main/teleop/run_g1_control_loop.py`
- [G1 environment](https://github.com/nvlabs/gr00t-wholebodycontrol/blob/main/decoupled_wbc/control/envs/g1/g1_env.py): `decoupled_wbc/control/envs/g1/g1_env.py`
- [Simulator selection](https://github.com/nvlabs/gr00t-wholebodycontrol/blob/main/decoupled_wbc/control/envs/g1/sim/simulator_factory.py): `decoupled_wbc/control/envs/g1/sim/simulator_factory.py`
- [G1 simulators](https://github.com/nvlabs/gr00t-wholebodycontrol/blob/main/decoupled_wbc/control/envs/g1/sim/robocasa_sim.py): `decoupled_wbc/control/envs/g1/sim/robocasa_sim.py`
- [Data export](https://github.com/nvlabs/gr00t-wholebodycontrol/blob/main/decoupled_wbc/data/exporter.py): `decoupled_wbc/data/exporter.py`

### GEAR-SONIC

- [SONIC environment wrapper](https://github.com/nvlabs/gr00t-wholebodycontrol/blob/main/gear_sonic/envs/wrapper/manager_env_wrapper.py): `gear_sonic/envs/wrapper/manager_env_wrapper.py`
- [SONIC TensorRT inference](https://github.com/nvlabs/gr00t-wholebodycontrol/blob/main/gear_sonic_deploy/src/TRTInference/InferenceEngine.cpp): `gear_sonic_deploy/src/TRTInference/InferenceEngine.cpp`
- [ZMQ motion input](https://github.com/nvlabs/gr00t-wholebodycontrol/blob/main/gear_sonic_deploy/src/g1/g1_deploy_onnx_ref/include/input_interface/zmq_manager.hpp): `gear_sonic_deploy/src/g1/g1_deploy_onnx_ref/include/input_interface/zmq_manager.hpp`
- [Gamepad input](https://github.com/nvlabs/gr00t-wholebodycontrol/blob/main/gear_sonic_deploy/src/g1/g1_deploy_onnx_ref/include/input_interface/gamepad_manager.hpp): `gear_sonic_deploy/src/g1/g1_deploy_onnx_ref/include/input_interface/gamepad_manager.hpp`
- [SONIC robot deployment](https://github.com/nvlabs/gr00t-wholebodycontrol/tree/main/gear_sonic_deploy/src/g1): `gear_sonic_deploy/src/g1`

### MotionBricks

- [Interactive motion controllers](https://github.com/nvlabs/gr00t-wholebodycontrol/blob/main/motionbricks/motionbricks/motion_backbone/demo/controllers.py): `motionbricks/motionbricks/motion_backbone/demo/controllers.py`

### Other components

- **Operator**
- **G1 robot**

## Connections

- Operator → Teleop devices and UI: operates
- Teleop devices and UI → Teleop streamers: starts streams
- Teleop streamers → Retargeting and IK: streams poses
- Operator → Control and simulation loops: selects control
- Control and simulation loops → G1 WBC policies: runs policy
- Control and simulation loops → G1 environment: steps environment
- G1 environment → Simulator selection: selects simulator
- Simulator selection → G1 simulators: creates simulator
- Control and simulation loops → G1 robot: controls robot
- Control and simulation loops → Data export: exports data
- SONIC environment wrapper → SONIC TensorRT inference: decodes actions
- Operator → Gamepad input: operates
- Operator → ZMQ motion input: streams commands
- Gamepad input → SONIC robot deployment: selects input
- ZMQ motion input → SONIC robot deployment: provides motion
- SONIC TensorRT inference → SONIC robot deployment: infers control
- Operator → Interactive motion controllers: sets motion