How marineroboticsgroup/opti-acoustic-semantics works
Opti-Acoustic Semantic Mapping combines visual object segmentation or detection with sonar/depth geometry and ROS outputs for semantic mapping. Its main runtime path turns synchronized camera/depth or camera/sonar observations into object vectors and visualization topics; a TF adapter supplies odometry to the mapping consumer. The repository also includes offline ViT-feature workflows for co-segmentation, image correspondences, PCA, similarity inspection, and interactive object selection. Source excerpts are partial, so uncertain internal call wiring is omitted.
Main entry points: extractor.py ViT extractor · image_sonar_syncing_node.py Sonar sync · image_segmentation_node.py Image segmenter · yolov8_detector.py YOLO detector
Components 16
ROS perception
- Image segmenter
image_segmentation_node.py - Sonar sync
image_sonar_syncing_node.py - YOLO detector
yolov8_detector.py - Odometry adapter
tf_to_odom_node.py
Visual features
- ViT extractor
extractor.py - Co-segmentation
cosegmentation.py - Correspondences
correspondences.py - Feature PCA
pca.py - Similarity inspection
inspect_similarity.py - Object selection
object_selection_gui.py
Mapping inputs
- Object vectors
Other components
- Camera / depth
- Oculus sonar
- TF source
- DCSAM mapping
- sonar_oculus
Connections 16
- Camera / depth → Image segmenter: sends images
- Camera / depth → Sonar sync: sends images
- Camera / depth → YOLO detector: sends images
- Oculus sonar → Sonar sync: sends pings
- sonar_oculus → Oculus sonar: provides sonar
- Image segmenter → Object vectors: publishes objects
- Sonar sync → Object vectors: publishes objects
- YOLO detector → Object vectors: publishes objects
- Object vectors → DCSAM mapping: feeds mapping
- TF source → Odometry adapter: sends transforms
- Odometry adapter → DCSAM mapping: publishes pose
- ViT extractor → Co-segmentation: extracts descriptors
- ViT extractor → Correspondences: extracts descriptors
- ViT extractor → Feature PCA: extracts descriptors
- ViT extractor → Similarity inspection: extracts descriptors
- ViT extractor → Object selection: extracts descriptors