How leoxie080808/ringfusion works
RingFusion turns a rectified wide-angle camera image and sparse TMF8829 ranges into metric depth, uncertainty, and a point cloud. The live workflow is sensor acquisition, ToF-to-camera projection, disparity anchoring and closed-form scale fitting, then optional learned refinement and ROS output. Firmware streams ToF frames; ROS bringup starts the camera, ToF driver, and perception node. Training and model export form an offline branch. The graph follows README capabilities and sampled pipeline/launch code; wiring for unsampled ROS internals is shown only where documented by launch or README.
Main entry points: perception_node.py Perception node · pipeline.py Fusion pipeline · tof_driver_node.py ToF driver · single_module.launch.py ROS bringup
Components 24
Sensor acquisition
- ESP32 firmware
firmware-esp/main/main.c - TMF8829 driver
firmware-esp/components/tmf8829/tmf8829.c - Camera driver
ros2_ws/src/ringfusion_drivers/ringfusion_drivers/camera_node.py - ToF driver
ros2_ws/src/ringfusion_drivers/ringfusion_drivers/tof_driver_node.py - ToF stream parser
ros2_ws/src/ringfusion_drivers/ringfusion_drivers/tof_source.py
ROS perception
- ROS bringup
ros2_ws/src/ringfusion_bringup/launch/single_module.launch.py - Image rectification
ros2_ws/src/ringfusion_perception/ringfusion_perception/rectify.py - Perception node
ros2_ws/src/ringfusion_perception/ringfusion_perception/perception_node.py - Fusion pipeline
ros2_ws/src/ringfusion_perception/ringfusion_perception/pipeline.py - ToF projection
ros2_ws/src/ringfusion_perception/ringfusion_perception/geometry.py - Affine anchoring
ros2_ws/src/ringfusion_perception/ringfusion_perception/anchoring.py - Depth blend
ros2_ws/src/ringfusion_perception/ringfusion_perception/blend.py - ROI plane mask
ros2_ws/src/ringfusion_perception/ringfusion_perception/roi.py - Cloud conversion
ros2_ws/src/ringfusion_perception/ringfusion_perception/cloud_util.py
Depth models
- Disparity backbone
ros2_ws/src/ringfusion_perception/ringfusion_perception/backbone.py - Residual refiner
ros2_ws/src/ringfusion_perception/ringfusion_perception/residual.py - TensorRT runtime
ros2_ws/src/ringfusion_perception/ringfusion_perception/trt_util.py
Training and export
- Model training
training - Student network
training/models/student.py - ONNX export
tools/export_onnx.py - Camera calibration
tools/calibrate_camera.py
Other components
- Wide camera
- TMF8829 sensor
- ROS consumers
Connections 26
- TMF8829 sensor → TMF8829 driver: ranges sensor
- ESP32 firmware → TMF8829 driver: calls driver
- ESP32 firmware → ToF driver: streams frames
- ToF driver → ToF stream parser: reads serial
- Wide camera → Camera driver: captures images
- ROS bringup → Camera driver: launches
- ROS bringup → ToF driver: launches
- ROS bringup → Perception node: launches
- Camera driver → Perception node: publishes images
- ToF driver → Perception node: publishes ranges
- Perception node → Image rectification: rectifies input
- Perception node → Fusion pipeline: runs frame
- Fusion pipeline → Disparity backbone: infers disparity
- Fusion pipeline → ToF projection: projects zones
- Fusion pipeline → Affine anchoring: fits scale
- Fusion pipeline → Depth blend: blends depth
- Fusion pipeline → ROI plane mask: masks regions
- Fusion pipeline → Residual refiner: refines depth
- Perception node → Cloud conversion: builds cloud
- Perception node → ROS consumers: publishes depth cloud
- Model training → Student network: trains network
- Student network → ONNX export: exports weights
- ONNX export → TensorRT runtime: creates ONNX
- TensorRT runtime → Disparity backbone: runs inference
- TensorRT runtime → Residual refiner: runs inference
- Camera calibration → Perception node: supplies calibration