How wuphilipp/gello_software works
GELLO is a robot-manipulator teleoperation framework. Its main loop turns a leader device’s input into joint commands for a configured real or simulated robot, while RobotEnv collects robot and camera observations. The repository also contains simulation tasks, demonstration-data conversion, standalone FACTR gravity compensation, and a ROS 2 Franka FR3 implementation. The README documents YAML-based and non-YAM launches and supported robot families; unsampled subsystems are represented at their existing directory paths, with links omitted where source evidence does not establish a call.
Main entry points: env.py Robot environment · launch_utils.py Launch orchestration · gravity_compensation.py FACTR gravity compensation · camera.py Camera interface
Components 33
Teleoperation
- Launch orchestration
gello/utils/launch_utils.py - Control loop
gello/utils/control_utils.py - Agent interface
gello/agents/agent.py - GELLO agent
gello/agents/gello_agent.py - SpaceMouse agent
gello/agents/spacemouse_agent.py - Quest agent
gello/agents/quest_agent.py
Robot and Sensors
- Robot environment
gello/env.py - Robot interface
gello/robots/robot.py - Robot drivers
gello/robots - Dynamixel driver
gello/dynamixel/driver.py - Camera interface
gello/cameras/camera.py - RealSense camera
gello/cameras/realsense_camera.py - Robot ZMQ server/client
gello/zmq_core/robot_node.py - Camera ZMQ node
gello/zmq_core/camera_node.py
Simulation and Data
- Simulation arms
gello/dm_control_tasks/arms/manipulator.py - Manipulation tasks
gello/dm_control_tasks/manipulation/tasks - Simulation arenas
gello/dm_control_tasks/manipulation/arenas - Mujoco robot
gello/robots/sim_robot.py - Demo conversion
gello/data_utils/demo_to_gdict.py - Conversion utilities
gello/data_utils/conversion_utils.py - MJCF utilities
gello/dm_control_tasks/mjcf_utils.py
FACTR Compensation
- FACTR gravity compensation
gello/factr/gravity_compensation.py - FACTR YAM agent
gello/factr/yam_gello_factr_agent.py
ROS 2 Franka
- GELLO state publisher
ros2/src/franka_gello_state_publisher/franka_gello_state_publisher/gello_publisher.py - Franka GELLO hardware
ros2/src/franka_gello_state_publisher/franka_gello_state_publisher/gello_hardware.py - Joint impedance controller
ros2/src/franka_fr3_arm_controllers/src/joint_impedance_controller.cpp - Motion generator
ros2/src/franka_fr3_arm_controllers/src/motion_generator.cpp - Franka gripper client
ros2/src/franka_gripper_manager/franka_gripper_manager/franka_gripper_client.py - Robotiq gripper client
ros2/src/franka_gripper_manager/franka_gripper_manager/robotiq_gripper_client.py
Other components
- Operator
- Robot arm
- Camera device
- Franka FR3
Connections 31
- Operator → Launch orchestration: starts
README.md - Launch orchestration → Control loop: runs
gello/utils/launch_utils.py - Control loop → Agent interface: requests action
gello/utils/control_utils.py - GELLO agent → Agent interface: implements
gello/agents/agent.py - SpaceMouse agent → Agent interface: implements
gello/agents/spacemouse_agent.py - Quest agent → Agent interface: implements
gello/agents/quest_agent.py - Control loop → Robot environment: steps
gello/utils/control_utils.py - Robot environment → Robot interface: commands and reads
gello/env.py - Robot environment → Camera interface: reads frames
gello/env.py - RealSense camera → Camera interface: implements
gello/cameras/realsense_camera.py - Robot drivers → Robot interface: implement
gello/robots/xarm_robot.py - Robot drivers → Dynamixel driver: uses
gello/robots/dynamixel.py - Robot ZMQ server/client → Robot interface: dispatches methods
gello/zmq_core/robot_node.py - Camera ZMQ node → Camera interface: uses
gello/zmq_core/camera_node.py - Launch orchestration → Robot ZMQ server/client: configures
gello/utils/launch_utils.py - Launch orchestration → Dynamixel driver: initializes
gello/utils/launch_utils.py - Robot environment → Robot arm: commands via adapter
- Camera interface → Camera device: reads from
- Manipulation tasks → Simulation arms: uses
gello/dm_control_tasks/manipulation/tasks/block_play.py - Manipulation tasks → Simulation arenas: uses
gello/dm_control_tasks/manipulation/tasks/block_play.py - Simulation arms → MJCF utilities: uses MJCF helpers
gello/dm_control_tasks/arms/manipulator.py - Demo conversion → Conversion utilities: uses
gello/data_utils/demo_to_gdict.py - FACTR gravity compensation → Dynamixel driver: uses
gello/factr/gravity_compensation.py - FACTR gravity compensation → Robot ZMQ server/client: optionally connects
gello/factr/gravity_compensation.py - FACTR gravity compensation → Robot environment: optionally creates
gello/factr/gravity_compensation.py - FACTR YAM agent → FACTR gravity compensation: uses
gello/factr/yam_gello_factr_agent.py - GELLO state publisher → Franka GELLO hardware: uses hardware interface
- Franka GELLO hardware → Franka FR3: interfaces with
- Joint impedance controller → Motion generator: uses
- Franka gripper client → Franka FR3: controls gripper
- Robotiq gripper client → Franka FR3: controls gripper