# marineroboticsgroup/opti-acoustic-semantics architecture

> Architecture diagram of the GitHub repository marineroboticsgroup/opti-acoustic-semantics, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-09-29). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/marineroboticsgroup/opti-acoustic-semantics
- Repository: https://github.com/marineroboticsgroup/opti-acoustic-semantics

## Overview

Opti-Acoustic Semantic Mapping combines visual object segmentation or detection with sonar/depth geometry and ROS outputs for semantic mapping. Its main runtime path turns synchronized camera/depth or camera/sonar observations into object vectors and visualization topics; a TF adapter supplies odometry to the mapping consumer. The repository also includes offline ViT-feature workflows for co-segmentation, image correspondences, PCA, similarity inspection, and interactive object selection. Source excerpts are partial, so uncertain internal call wiring is omitted.

Main entry points: [`extractor.py`](https://github.com/marineroboticsgroup/opti-acoustic-semantics/blob/master/extractor.py) (ViT extractor), [`image_sonar_syncing_node.py`](https://github.com/marineroboticsgroup/opti-acoustic-semantics/blob/master/image_sonar_syncing_node.py) (Sonar sync), [`image_segmentation_node.py`](https://github.com/marineroboticsgroup/opti-acoustic-semantics/blob/master/image_segmentation_node.py) (Image segmenter), [`yolov8_detector.py`](https://github.com/marineroboticsgroup/opti-acoustic-semantics/blob/master/yolov8_detector.py) (YOLO detector)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_runtime["ROS perception"]
  node_seg_node["Image segmenter"]
  node_sonar_node["Sonar sync"]
  node_yolo["YOLO detector<br/>[yolov8_detector.py]"]
  node_odom["Odometry adapter<br/>[tf_to_odom_node.py]"]
end

subgraph group_features["Visual features"]
  node_extractor["ViT extractor<br/>[extractor.py]"]
  node_coseg["Co-segmentation<br/>[cosegmentation.py]"]
  node_correspondences["Correspondences<br/>[correspondences.py]"]
  node_pca["Feature PCA<br/>[pca.py]"]
  node_similarity["Similarity inspection"]
  node_selection["Object selection"]
end

subgraph group_mapping["Mapping inputs"]
  node_objects["Object vectors"]
end

node_camera(("Camera / depth"))
node_sonar(("Oculus sonar"))
node_tf(("TF source"))
node_mapping_system["DCSAM mapping"]
node_sonar_driver["sonar_oculus"]

node_camera -->|"sends images"| node_seg_node
node_camera -->|"sends images"| node_sonar_node
node_camera -->|"sends images"| node_yolo
node_sonar -->|"sends pings"| node_sonar_node
node_sonar_driver -.->|"provides sonar"| node_sonar
node_seg_node -->|"publishes objects"| node_objects
node_sonar_node -->|"publishes objects"| node_objects
node_yolo -->|"publishes objects"| node_objects
node_objects -.->|"feeds mapping"| node_mapping_system
node_tf -->|"sends transforms"| node_odom
node_odom -->|"publishes pose"| node_mapping_system
node_extractor -->|"extracts descriptors"| node_coseg
node_extractor -->|"extracts descriptors"| node_correspondences
node_extractor -->|"extracts descriptors"| node_pca
node_extractor -->|"extracts descriptors"| node_similarity
node_extractor -->|"extracts descriptors"| node_selection

click node_seg_node "https://github.com/marineroboticsgroup/opti-acoustic-semantics/blob/master/image_segmentation_node.py"
click node_sonar_node "https://github.com/marineroboticsgroup/opti-acoustic-semantics/blob/master/image_sonar_syncing_node.py"
click node_yolo "https://github.com/marineroboticsgroup/opti-acoustic-semantics/blob/master/yolov8_detector.py"
click node_odom "https://github.com/marineroboticsgroup/opti-acoustic-semantics/blob/master/tf_to_odom_node.py"
click node_extractor "https://github.com/marineroboticsgroup/opti-acoustic-semantics/blob/master/extractor.py"
click node_coseg "https://github.com/marineroboticsgroup/opti-acoustic-semantics/blob/master/cosegmentation.py"
click node_correspondences "https://github.com/marineroboticsgroup/opti-acoustic-semantics/blob/master/correspondences.py"
click node_pca "https://github.com/marineroboticsgroup/opti-acoustic-semantics/blob/master/pca.py"
click node_similarity "https://github.com/marineroboticsgroup/opti-acoustic-semantics/blob/master/inspect_similarity.py"
click node_selection "https://github.com/marineroboticsgroup/opti-acoustic-semantics/blob/master/object_selection_gui.py"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_seg_node,node_sonar_node,node_yolo,node_odom toneBlue
class node_extractor,node_coseg,node_correspondences,node_pca,node_similarity,node_selection toneAmber
class node_objects toneMint
class node_camera,node_sonar,node_tf,node_mapping_system,node_sonar_driver toneIndigo
```

## Components

### ROS perception

- [Image segmenter](https://github.com/marineroboticsgroup/opti-acoustic-semantics/blob/master/image_segmentation_node.py): `image_segmentation_node.py`
- [Sonar sync](https://github.com/marineroboticsgroup/opti-acoustic-semantics/blob/master/image_sonar_syncing_node.py): `image_sonar_syncing_node.py`
- [YOLO detector](https://github.com/marineroboticsgroup/opti-acoustic-semantics/blob/master/yolov8_detector.py): `yolov8_detector.py`
- [Odometry adapter](https://github.com/marineroboticsgroup/opti-acoustic-semantics/blob/master/tf_to_odom_node.py): `tf_to_odom_node.py`

### Visual features

- [ViT extractor](https://github.com/marineroboticsgroup/opti-acoustic-semantics/blob/master/extractor.py): `extractor.py`
- [Co-segmentation](https://github.com/marineroboticsgroup/opti-acoustic-semantics/blob/master/cosegmentation.py): `cosegmentation.py`
- [Correspondences](https://github.com/marineroboticsgroup/opti-acoustic-semantics/blob/master/correspondences.py): `correspondences.py`
- [Feature PCA](https://github.com/marineroboticsgroup/opti-acoustic-semantics/blob/master/pca.py): `pca.py`
- [Similarity inspection](https://github.com/marineroboticsgroup/opti-acoustic-semantics/blob/master/inspect_similarity.py): `inspect_similarity.py`
- [Object selection](https://github.com/marineroboticsgroup/opti-acoustic-semantics/blob/master/object_selection_gui.py): `object_selection_gui.py`

### Mapping inputs

- **Object vectors**

### Other components

- **Camera / depth**
- **Oculus sonar**
- **TF source**
- **DCSAM mapping**
- **sonar_oculus**

## Connections

- Camera / depth → Image segmenter: sends images
- Camera / depth → Sonar sync: sends images
- Camera / depth → YOLO detector: sends images
- Oculus sonar → Sonar sync: sends pings
- sonar_oculus → Oculus sonar: provides sonar
- Image segmenter → Object vectors: publishes objects
- Sonar sync → Object vectors: publishes objects
- YOLO detector → Object vectors: publishes objects
- Object vectors → DCSAM mapping: feeds mapping
- TF source → Odometry adapter: sends transforms
- Odometry adapter → DCSAM mapping: publishes pose
- ViT extractor → Co-segmentation: extracts descriptors
- ViT extractor → Correspondences: extracts descriptors
- ViT extractor → Feature PCA: extracts descriptors
- ViT extractor → Similarity inspection: extracts descriptors
- ViT extractor → Object selection: extracts descriptors