How seriouslysilly21612/mpsoc-paper works
This is an Indy7 camera-guided picking application for a Kria KV260: ROS 2 vision detects objects, chooses a target, estimates its 3-D pose in base_link, and hands pick goals to the robot controller. The controller runs EtherCAT, trajectory, and computed-torque tasks, with a MuJoCo simulation alternative; an operator GUI exposes video, robot state, and controls. Calibration, training, and analysis tools support development. The README documents the broad workflow; several ROS nodes and bridge implementations were not excerpted, so their detailed message-level links are not asserted.
Main entry points: Indy7Ctrl.h Indy7 task coordinator · vitis_ai_detector_node.py YOLO detector · main.cpp Operator console · pick_logic.py Pick selection
Components 17
Camera perception
- Camera input
- YOLO detector
ros2_ws/src/vitis_ai_detector_pkg/vitis_ai_detector_pkg/vitis_ai_detector_node.py - DPU worker
ros2_ws/src/vitis_ai_detector_pkg/vitis_ai_detector_pkg/vitis_ai_worker_yolo.py - Pick selection
ros2_ws/src/pick_logic_pkg/pick_logic_pkg/pick_logic.py - 3D localization
ros2_ws/src/target_3d_pkg/target_3d_pkg/pick_target_3d_node.py - Base target
ros2_ws/src/target_3d_pkg/target_3d_pkg/pick_target_base_node.py
Robot control
- Controller entry
goal-to-torque_MPSoC/indy7/src/main.cpp - Indy7 task coordinator
goal-to-torque_MPSoC/indy7/src/Indy7Ctrl.h - Computed torque
goal-to-torque_MPSoC/indy7/src/FullDynControllerRT.cpp - EtherCAT arm
goal-to-torque_MPSoC/indy7/src/ecat/EcatMaster.cpp - MuJoCo arm
goal-to-torque_MPSoC/indy7/src/sim/SimArm.cpp - ROS 2 bridge
goal-to-torque_MPSoC/indy7/src/ROS2PickBridge.h
Operator interface
- Operator console
ros2_ws/src/raon_operator_gui/src/main.cpp
Calibration and analysis
- Hand-eye calibration
goal-to-torque_MPSoC/calib/eye_to_hand_calib.py - YOLO training
ros2_ws/yolo_v3_tiny_training/scripts/12_quantize.py - Log analysis
goal-to-torque_MPSoC/tools/hl_analyze.py
Other components
- Robot operator
Connections 14
- Camera input → YOLO detector: supplies frames
README.md - YOLO detector → DPU worker: dispatches inference
ros2_ws/src/vitis_ai_detector_pkg/vitis_ai_detector_pkg/vitis_ai_worker_yolo.py - YOLO detector → Pick selection: publishes detections
README.md - Pick selection → 3D localization: selects target
README.md - 3D localization → Base target: localizes in base_link
README.md - Base target → ROS 2 bridge: provides pick goal
- Robot operator → Controller entry: starts controller
README.md - Controller entry → Indy7 task coordinator: initializes
goal-to-torque_MPSoC/indy7/src/main.cpp - Indy7 task coordinator → Computed torque: runs control task
goal-to-torque_MPSoC/indy7/src/Indy7Ctrl.h - Indy7 task coordinator → EtherCAT arm: runs bus task
README.md - Indy7 task coordinator → MuJoCo arm: selects simulation
- ROS 2 bridge → Indy7 task coordinator: forwards commands
goal-to-torque_MPSoC/indy7/src/Indy7Ctrl.h - Operator console → Camera input: shows video
- Operator console → Indy7 task coordinator: operator commands