How nvlabs/gr00t-wholebodycontrol works
This repository brings together humanoid whole-body control, teleoperation, simulation and deployment, plus the SONIC and MotionBricks projects described in the README. A principal workflow is operator or streamed input through teleoperation/control components toward a simulated or deployed robot, with data-export tooling supporting collection. The tree and excerpts identify these subsystems, but do not establish all their runtime wiring; edges are limited to documented behavior and sampled calls. SONIC training and MotionBricks capabilities are README-level context, not inspected implementation paths.
Main entry points: run_g1_control_loop.py Control and simulation loops · g1 SONIC robot deployment · main.py Teleop devices and UI · teleop_streamer.py Teleop streamers
Components 17
Teleoperation
- Teleop devices and UI
decoupled_wbc/control/teleop/gui/main.py - Teleop streamers
decoupled_wbc/control/teleop/teleop_streamer.py - Retargeting and IK
decoupled_wbc/control/teleop/teleop_retargeting_ik.py
Decoupled WBC
- G1 WBC policies
decoupled_wbc/control/policy/g1_decoupled_whole_body_policy.py - Control and simulation loops
decoupled_wbc/control/main/teleop/run_g1_control_loop.py - G1 environment
decoupled_wbc/control/envs/g1/g1_env.py - Simulator selection
decoupled_wbc/control/envs/g1/sim/simulator_factory.py - G1 simulators
decoupled_wbc/control/envs/g1/sim/robocasa_sim.py - Data export
decoupled_wbc/data/exporter.py
GEAR-SONIC
- SONIC environment wrapper
gear_sonic/envs/wrapper/manager_env_wrapper.py - SONIC TensorRT inference
gear_sonic_deploy/src/TRTInference/InferenceEngine.cpp - ZMQ motion input
gear_sonic_deploy/src/g1/g1_deploy_onnx_ref/include/input_interface/zmq_manager.hpp - Gamepad input
gear_sonic_deploy/src/g1/g1_deploy_onnx_ref/include/input_interface/gamepad_manager.hpp - SONIC robot deployment
gear_sonic_deploy/src/g1
MotionBricks
- Interactive motion controllers
motionbricks/motionbricks/motion_backbone/demo/controllers.py
Other components
- Operator
- G1 robot
Connections 17
- Operator → Teleop devices and UI: operates
- Teleop devices and UI → Teleop streamers: starts streams
- Teleop streamers → Retargeting and IK: streams poses
- Operator → Control and simulation loops: selects control
- Control and simulation loops → G1 WBC policies: runs policy
- Control and simulation loops → G1 environment: steps environment
- G1 environment → Simulator selection: selects simulator
- Simulator selection → G1 simulators: creates simulator
- Control and simulation loops → G1 robot: controls robot
- Control and simulation loops → Data export: exports data
- SONIC environment wrapper → SONIC TensorRT inference: decodes actions
- Operator → Gamepad input: operates
- Operator → ZMQ motion input: streams commands
- Gamepad input → SONIC robot deployment: selects input
- ZMQ motion input → SONIC robot deployment: provides motion
- SONIC TensorRT inference → SONIC robot deployment: infers control
- Operator → Interactive motion controllers: sets motion