How dionyichia/gappler works
This repository is a ROS 2 robotics stack spanning Project Aria sensing, object perception, grasp planning, and mobile navigation. The principal intended workflow is to acquire visual/gaze data, identify a prompted object, and make perception available to grasp and robot-control components. The sampled Aria application has an explicit runtime orchestrator; links between separate ROS packages are shown only where the supplied source index or excerpts support them. Several package-level capabilities are visible in the tree but their runtime wiring was not sampled.
Main entry points: main.py Aria application · object_recognition_pipeline.py Object recognition · gappler_common.py Shared configuration · grasp_orchestrator.py Grasp orchestrator
Components 26
Aria sensing
- Aria application
aria/aria_app/main.py - Aria device control
aria/aria_app/services/aria_device/aria_device_controller.py - Image pipeline
aria/aria_app/services/aria_device/stream/image_streaming_pipeline.py - Audio pipeline
aria/aria_app/services/aria_device/stream/audio_streaming_pipeline.py - VIO pose pipeline
aria/aria_app/services/aria_device/stream/pose_streaming_pipeline.py - Recording playback
aria/aria_app/services/playback_controller.py - Prompt extraction
aria/aria_app/services/prompt_extractor.py
Perception
- Object recognition
aria/aria_app/services/object_recognition/object_recognition_pipeline.py - SAM 3 model
aria/aria_app/services/object_recognition/sam3_model.py - Feature matching
aria/aria_app/services/feature_matching.py - Mask visualization
aria/aria_app/services/visualizer/renderers/object_mask_visualizer.py - ROS visualization feeds
aria/aria_app/services/visualizer/ros_manager.py
Grasp execution
- Grasp orchestrator
launchers/grasp_orchestrator.py - Grasp pipeline launcher
launchers/start_grasp_pipeline.py - SAM ROS node
grasp/segmentation/sam3_ros_node.py - AnyGrasp detection
grasp/anygrasp_node/anygrasp_node.py - Grasp state machine
grasp/grasp_state_machine/src/grasp_state_machine.cpp - MTC planner
grasp/grasp_state_machine/src/mtc_planner.cpp - Arm bringup
arm/arm_bringup/launch/arm_bringup.launch.py - Emergency stop
arm/estop/estop.py
Navigation
- Navigation launch
nav/robot_navigation/launch/navigation.launch.py - SLAM localization
nav/robot_slam/launch/slam_localization.launch.py - Object approach
nav/object_approach/object_approach_node.py - Navigate to glasses
nav/goto_glasses/goto_glasses.py
Shared runtime
- Shared configuration
shared/gappler_common.py
Other components
- Operator
Connections 20
- Operator → Aria application: starts
- Aria application → Aria device control: starts stream
aria/aria_app/main.py - Aria application → Audio pipeline: starts audio
aria/aria_app/main.py - Aria application → Object recognition: configures
aria/aria_app/main.py - Aria application → Feature matching: configures
aria/aria_app/main.py - Aria application → Recording playback: starts playback
aria/aria_app/main.py - Aria application → Image pipeline: starts visual feed
aria/aria_app/main.py - Aria application → VIO pose pipeline: starts pose feed
aria/aria_app/main.py - Audio pipeline → Prompt extraction: extracts intent
aria/aria_app/services/aria_device/stream/audio_streaming_pipeline.py - Object recognition → Feature matching: uses matching
aria/aria_app/services/object_recognition/object_recognition_pipeline.py - Object recognition → Mask visualization: uses renderer
aria/aria_app/services/object_recognition/object_recognition_pipeline.py - ROS visualization feeds → Operator: delivers visualization
- SAM ROS node → SAM 3 model: uses model
grasp/segmentation/sam3_ros_node.py - SAM ROS node → Mask visualization: uses renderer
grasp/segmentation/sam3_ros_node.py - Grasp state machine → MTC planner: uses planner
grasp/grasp_state_machine/src/grasp_state_machine.cpp - Grasp orchestrator → Shared configuration: reads paths
launchers/grasp_orchestrator.py - Grasp pipeline launcher → Shared configuration: reads paths
launchers/start_grasp_pipeline.py - SLAM localization → Shared configuration: reads paths
nav/robot_slam/launch/slam_localization.launch.py - Navigation launch → SLAM localization: launches localization
- Grasp orchestrator → Grasp pipeline launcher: coordinates