How ros-physical-ai/demos works
ROS Physical AI Demos provides ROS 2 SO-ARM101 robot demos across Gazebo, MuJoCo, and physical hardware, plus a LeRobot-based Record → Train → Deploy workflow. An operator brings up a robot, chooses a teleoperation or inference input, and sends commands through ROS control to the arm. Demonstration collection feeds the learning workflow; visualization and optional MCP access support inspection and control. The README documents several capabilities whose implementations are external or not sampled here, so those are represented at subsystem level; edges reflect documented launch and topic relationships rather than inferred internal calls.
Main entry points: ros2_controllers.yaml ROS control · so_arm_gz_bringup.launch.py Robot bringup · rosetta Rosetta bridge · visualizer_node.py Rerun visualizer
Components 24
Robot runtime
- Robot bringup
pai_bringup/launch/so_arm_gz_bringup.launch.py - Common bringup
pai_bringup/launch/include/so_arm_common.launch.py - Robot and scene
pai_description/world/so_arm_table.sdf - Gazebo / MuJoCo
pai_bringup/launch - Physical arm
pai_bringup/urdf/so_arm_real.urdf.xacro - ROS control
pai_bringup/config/control/ros2_controllers.yaml - Camera stack
pai_bringup/launch/include/cameras.launch.py - Scene assets
pai_assets/models
Teleoperation
- Leader teleop
pai_leader_teleop/launch/leader_bringup.launch.py - Leader command relay
pai_leader_teleop/scripts/leader_teleop_node - Phone adapter
pai_phone_teleop/pai_phone_teleop/phone_teleop_node.py - RViz marker adapter
pai_rviz_teleop/launch/interactive_ik.launch.py - Cartesian IK servo
pai_teleop_ik/pai_teleop_ik/servo_node.py
Learning pipeline
- Rosetta bridge
pai_data_collection/config/rosetta - Demonstration dataset
- LeRobot train / policy
- Policy inference
pai_data_collection/config/rosetta/policy_runner.yaml
Inspection and access
- Rerun visualizer
pai_rerun_visualizer/pai_rerun_visualizer/visualizer_node.py - ROS bridge
pai_bringup/launch/include/so_arm_common.launch.py
Other components
- Operator
- Phone user
- MCP client
- Rerun viewer
- ROS-MCP server
Connections 32
- Operator → Robot bringup: starts
- Robot bringup → Common bringup: includes
- Robot bringup → Gazebo / MuJoCo: starts Gazebo
- Common bringup → Robot and scene: loads description
- Gazebo / MuJoCo → Robot and scene: loads scene
- Robot and scene → Scene assets: references models
- Common bringup → ROS control: spawns controllers
- Robot bringup → Camera stack: includes cameras
- Operator → Leader teleop: starts
- Leader teleop → Leader command relay: launches
- Leader command relay → ROS control: publishes commands
- Operator → Phone adapter: starts
- Phone user → Phone adapter: sends WebXR pose
- Operator → RViz marker adapter: starts
- Phone adapter → Cartesian IK servo: publishes target pose
- RViz marker adapter → Cartesian IK servo: publishes target pose
- Cartesian IK servo → ROS control: publishes joint commands
- Operator → Rosetta bridge: starts collection
- Leader command relay → Rosetta bridge: provides demonstrations
- Camera stack → Rosetta bridge: provides observations
- Rosetta bridge → Demonstration dataset: records episodes
- Demonstration dataset → LeRobot train / policy: trains policy
- LeRobot train / policy → Policy inference: provides trained policy
- Policy inference → ROS control: sends actions
- Operator → Rerun visualizer: enables visualization
- ROS control → Rerun visualizer: publishes robot state
- Camera stack → Rerun visualizer: publishes images
- Rerun visualizer → Rerun viewer: logs visualization
- Operator → ROS bridge: enables MCP access
- ROS bridge → ROS-MCP server: exposes ROS WebSocket
- MCP client → ROS-MCP server: requests robot access
- ROS-MCP server → ROS bridge: accesses ROS