# mmcza/semantic-mapping-for-autonomous-vehicles architecture

> Architecture diagram of the GitHub repository mmcza/semantic-mapping-for-autonomous-vehicles, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-10-01). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/mmcza/semantic-mapping-for-autonomous-vehicles
- Repository: https://github.com/mmcza/semantic-mapping-for-autonomous-vehicles

## Overview

This repository builds semantic point-cloud maps for autonomous-vehicle scenes, with a ROS2 workflow that synchronizes camera images and point clouds, runs ONNX segmentation, projects labels using camera/LiDAR transforms, filters and colors points, then publishes a cloud for OctoMap and RViz. Related capabilities include model training and evaluation, point-cloud analysis, and GroundingDINO/SAM-assisted annotation. ROS relationships are documented in the README; sampled source also shows the annotation app’s HTTP request and the LangSAM model calls. Other tool relationships are not inferred from names alone.

Main entry points: [`segmentation_node/src/segmentation_node.cpp`](https://github.com/mmcza/semantic-mapping-for-autonomous-vehicles/blob/main/segmentation_node/src/segmentation_node.cpp) (Segmentation node), [`utils/segmentation_with_sam/lang_sam/lang_sam.py`](https://github.com/mmcza/semantic-mapping-for-autonomous-vehicles/blob/main/utils/segmentation_with_sam/lang_sam/lang_sam.py) (Text-guided segmentation), [`segmentation_node/launch/segmentation_node_launch.py`](https://github.com/mmcza/semantic-mapping-for-autonomous-vehicles/blob/main/segmentation_node/launch/segmentation_node_launch.py) (Mapping launch), [`utils/segmentation_with_sam/lang_sam/server.py`](https://github.com/mmcza/semantic-mapping-for-autonomous-vehicles/blob/main/utils/segmentation_with_sam/lang_sam/server.py) (Inference server)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_mapping["Semantic mapping"]
  node_ros_node["Segmentation node"]
  node_ros_header["Node state and interfaces"]
  node_ros_launch["Mapping launch"]
  node_octomap["OctoMap server"]
  node_octomap_saver["Semantic map saver"]
  node_rviz["RViz"]
end

subgraph group_models["Segmentation models"]
  node_onnx["ONNX models"]
  node_trainer["WandB training<br/>[main_wandb.py]"]
  node_deeplab["DeepLab models<br/>[deeplab3.py]"]
  node_deeplab_model["DeepLab architecture<br/>[model_deeplab.py]"]
  node_pidnet["PIDNet model<br/>[pidnet_model.py]"]
  node_unet["Multi-encoder UNet"]
  node_mask_creator["Mask creation<br/>[create_masks_b5.py]"]
end

subgraph group_annotation["Annotation tools"]
  node_langsam["Text-guided segmentation<br/>[lang_sam.py]"]
  node_gdino["GroundingDINO<br/>[gdino.py]"]
  node_sam["SAM masks<br/>[sam.py]"]
  node_sam_server["Inference server<br/>[server.py]"]
  node_sam_app["Annotation web app<br/>[app.py]"]
  node_mask_gen["Training mask generator"]
  node_batch_mask_gen["Batch mask generator"]
end

subgraph group_analysis["Data and analysis"]
  node_viewer["Segmentation viewer<br/>[simple_viewer.py]"]
  node_point_viewer["Point-cloud viewer<br/>[viewer.py]"]
  node_point_processor["Point-cloud processing"]
  node_inference_eval["Inference benchmarks"]
  node_accuracy_eval["Cloud accuracy"]
  node_cloud_stats["Cloud statistics"]
  node_bag_images["Bag image extraction<br/>[ros2_bag_to_png.py]"]
end

node_operator(("Operator"))
node_camera(("RGB camera"))
node_lidar(("LiDAR point cloud"))
node_tf(("TF transforms"))

node_operator -->|"launches"| node_ros_launch
node_ros_launch -->|"starts"| node_ros_node
node_ros_launch -->|"starts"| node_octomap
node_ros_launch -->|"starts"| node_rviz
node_camera -->|"provides images"| node_ros_node
node_lidar -->|"provides cloud"| node_ros_node
node_tf -->|"provides transforms"| node_ros_node
node_ros_node -->|"runs inference"| node_onnx
node_ros_node -->|"publishes cloud"| node_octomap
node_octomap -->|"visualizes map"| node_rviz
node_ros_node -->|"uses interfaces"| node_ros_header
node_operator -.->|"submits prompt"| node_sam_app
node_sam_app -->|"posts prediction"| node_sam_server
node_sam_server -->|"predicts with"| node_langsam
node_langsam -->|"detects boxes"| node_gdino
node_langsam -->|"predicts masks"| node_sam
node_mask_gen -->|"generates annotations"| node_langsam
node_batch_mask_gen -->|"generates annotations"| node_langsam

click node_onnx "https://github.com/mmcza/semantic-mapping-for-autonomous-vehicles/tree/main/HF_segmentation/ONNX_trained"
click node_ros_node "https://github.com/mmcza/semantic-mapping-for-autonomous-vehicles/blob/main/segmentation_node/src/segmentation_node.cpp"
click node_ros_header "https://github.com/mmcza/semantic-mapping-for-autonomous-vehicles/blob/main/segmentation_node/include/segmentation_node/segmentation_node.hpp"
click node_ros_launch "https://github.com/mmcza/semantic-mapping-for-autonomous-vehicles/blob/main/segmentation_node/launch/segmentation_node_launch.py"
click node_octomap_saver "https://github.com/mmcza/semantic-mapping-for-autonomous-vehicles/blob/main/segmentation_node/src/semantic_octomap_saver.cpp"
click node_trainer "https://github.com/mmcza/semantic-mapping-for-autonomous-vehicles/blob/main/HF_segmentation/main_wandb.py"
click node_deeplab "https://github.com/mmcza/semantic-mapping-for-autonomous-vehicles/blob/main/utils/deeplab3.py"
click node_deeplab_model "https://github.com/mmcza/semantic-mapping-for-autonomous-vehicles/blob/main/utils/model_deeplab.py"
click node_pidnet "https://github.com/mmcza/semantic-mapping-for-autonomous-vehicles/blob/main/utils/pidnet_model.py"
click node_unet "https://github.com/mmcza/semantic-mapping-for-autonomous-vehicles/blob/main/utils/unet_multiple_encoders.py"
click node_mask_creator "https://github.com/mmcza/semantic-mapping-for-autonomous-vehicles/blob/main/HF_segmentation/create_masks_b5.py"
click node_langsam "https://github.com/mmcza/semantic-mapping-for-autonomous-vehicles/blob/main/utils/segmentation_with_sam/lang_sam/lang_sam.py"
click node_gdino "https://github.com/mmcza/semantic-mapping-for-autonomous-vehicles/blob/main/utils/segmentation_with_sam/lang_sam/models/gdino.py"
click node_sam "https://github.com/mmcza/semantic-mapping-for-autonomous-vehicles/blob/main/utils/segmentation_with_sam/lang_sam/models/sam.py"
click node_sam_server "https://github.com/mmcza/semantic-mapping-for-autonomous-vehicles/blob/main/utils/segmentation_with_sam/lang_sam/server.py"
click node_sam_app "https://github.com/mmcza/semantic-mapping-for-autonomous-vehicles/blob/main/utils/segmentation_with_sam/app.py"
click node_mask_gen "https://github.com/mmcza/semantic-mapping-for-autonomous-vehicles/blob/main/utils/segmentation_with_sam/generate_masks_for_training.py"
click node_batch_mask_gen "https://github.com/mmcza/semantic-mapping-for-autonomous-vehicles/blob/main/utils/segmentation_with_sam/generate_masks_for_training_with_batch_queries.py"
click node_viewer "https://github.com/mmcza/semantic-mapping-for-autonomous-vehicles/blob/main/HF_segmentation/simple_viewer.py"
click node_point_viewer "https://github.com/mmcza/semantic-mapping-for-autonomous-vehicles/blob/main/utils/viewer.py"
click node_point_processor "https://github.com/mmcza/semantic-mapping-for-autonomous-vehicles/blob/main/point_cloud_processor.py"
click node_inference_eval "https://github.com/mmcza/semantic-mapping-for-autonomous-vehicles/blob/main/utils/inference_time_calculation.py"
click node_accuracy_eval "https://github.com/mmcza/semantic-mapping-for-autonomous-vehicles/blob/main/utils/pointcloud_accuracy_calculation.py"
click node_cloud_stats "https://github.com/mmcza/semantic-mapping-for-autonomous-vehicles/blob/main/utils/pointcloud_statistics_calculation.py"
click node_bag_images "https://github.com/mmcza/semantic-mapping-for-autonomous-vehicles/blob/main/utils/ros2_bag_to_png.py"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_ros_node,node_ros_header,node_ros_launch,node_octomap,node_octomap_saver,node_rviz toneBlue
class node_onnx,node_trainer,node_deeplab,node_deeplab_model,node_pidnet,node_unet,node_mask_creator toneAmber
class node_langsam,node_gdino,node_sam,node_sam_server,node_sam_app,node_mask_gen,node_batch_mask_gen toneMint
class node_viewer,node_point_viewer,node_point_processor,node_inference_eval,node_accuracy_eval,node_cloud_stats,node_bag_images toneRose
class node_operator,node_camera,node_lidar,node_tf toneIndigo
```

## Components

### Semantic mapping

- [Segmentation node](https://github.com/mmcza/semantic-mapping-for-autonomous-vehicles/blob/main/segmentation_node/src/segmentation_node.cpp): `segmentation_node/src/segmentation_node.cpp`
- [Node state and interfaces](https://github.com/mmcza/semantic-mapping-for-autonomous-vehicles/blob/main/segmentation_node/include/segmentation_node/segmentation_node.hpp): `segmentation_node/include/segmentation_node/segmentation_node.hpp`
- [Mapping launch](https://github.com/mmcza/semantic-mapping-for-autonomous-vehicles/blob/main/segmentation_node/launch/segmentation_node_launch.py): `segmentation_node/launch/segmentation_node_launch.py`
- **OctoMap server**
- [Semantic map saver](https://github.com/mmcza/semantic-mapping-for-autonomous-vehicles/blob/main/segmentation_node/src/semantic_octomap_saver.cpp): `segmentation_node/src/semantic_octomap_saver.cpp`
- **RViz**

### Segmentation models

- [ONNX models](https://github.com/mmcza/semantic-mapping-for-autonomous-vehicles/tree/main/HF_segmentation/ONNX_trained): `HF_segmentation/ONNX_trained`
- [WandB training](https://github.com/mmcza/semantic-mapping-for-autonomous-vehicles/blob/main/HF_segmentation/main_wandb.py): `HF_segmentation/main_wandb.py`
- [DeepLab models](https://github.com/mmcza/semantic-mapping-for-autonomous-vehicles/blob/main/utils/deeplab3.py): `utils/deeplab3.py`
- [DeepLab architecture](https://github.com/mmcza/semantic-mapping-for-autonomous-vehicles/blob/main/utils/model_deeplab.py): `utils/model_deeplab.py`
- [PIDNet model](https://github.com/mmcza/semantic-mapping-for-autonomous-vehicles/blob/main/utils/pidnet_model.py): `utils/pidnet_model.py`
- [Multi-encoder UNet](https://github.com/mmcza/semantic-mapping-for-autonomous-vehicles/blob/main/utils/unet_multiple_encoders.py): `utils/unet_multiple_encoders.py`
- [Mask creation](https://github.com/mmcza/semantic-mapping-for-autonomous-vehicles/blob/main/HF_segmentation/create_masks_b5.py): `HF_segmentation/create_masks_b5.py`

### Annotation tools

- [Text-guided segmentation](https://github.com/mmcza/semantic-mapping-for-autonomous-vehicles/blob/main/utils/segmentation_with_sam/lang_sam/lang_sam.py): `utils/segmentation_with_sam/lang_sam/lang_sam.py`
- [GroundingDINO](https://github.com/mmcza/semantic-mapping-for-autonomous-vehicles/blob/main/utils/segmentation_with_sam/lang_sam/models/gdino.py): `utils/segmentation_with_sam/lang_sam/models/gdino.py`
- [SAM masks](https://github.com/mmcza/semantic-mapping-for-autonomous-vehicles/blob/main/utils/segmentation_with_sam/lang_sam/models/sam.py): `utils/segmentation_with_sam/lang_sam/models/sam.py`
- [Inference server](https://github.com/mmcza/semantic-mapping-for-autonomous-vehicles/blob/main/utils/segmentation_with_sam/lang_sam/server.py): `utils/segmentation_with_sam/lang_sam/server.py`
- [Annotation web app](https://github.com/mmcza/semantic-mapping-for-autonomous-vehicles/blob/main/utils/segmentation_with_sam/app.py): `utils/segmentation_with_sam/app.py`
- [Training mask generator](https://github.com/mmcza/semantic-mapping-for-autonomous-vehicles/blob/main/utils/segmentation_with_sam/generate_masks_for_training.py): `utils/segmentation_with_sam/generate_masks_for_training.py`
- [Batch mask generator](https://github.com/mmcza/semantic-mapping-for-autonomous-vehicles/blob/main/utils/segmentation_with_sam/generate_masks_for_training_with_batch_queries.py): `utils/segmentation_with_sam/generate_masks_for_training_with_batch_queries.py`

### Data and analysis

- [Segmentation viewer](https://github.com/mmcza/semantic-mapping-for-autonomous-vehicles/blob/main/HF_segmentation/simple_viewer.py): `HF_segmentation/simple_viewer.py`
- [Point-cloud viewer](https://github.com/mmcza/semantic-mapping-for-autonomous-vehicles/blob/main/utils/viewer.py): `utils/viewer.py`
- [Point-cloud processing](https://github.com/mmcza/semantic-mapping-for-autonomous-vehicles/blob/main/point_cloud_processor.py): `point_cloud_processor.py`
- [Inference benchmarks](https://github.com/mmcza/semantic-mapping-for-autonomous-vehicles/blob/main/utils/inference_time_calculation.py): `utils/inference_time_calculation.py`
- [Cloud accuracy](https://github.com/mmcza/semantic-mapping-for-autonomous-vehicles/blob/main/utils/pointcloud_accuracy_calculation.py): `utils/pointcloud_accuracy_calculation.py`
- [Cloud statistics](https://github.com/mmcza/semantic-mapping-for-autonomous-vehicles/blob/main/utils/pointcloud_statistics_calculation.py): `utils/pointcloud_statistics_calculation.py`
- [Bag image extraction](https://github.com/mmcza/semantic-mapping-for-autonomous-vehicles/blob/main/utils/ros2_bag_to_png.py): `utils/ros2_bag_to_png.py`

### Other components

- **Operator**
- **RGB camera**
- **LiDAR point cloud**
- **TF transforms**

## Connections

- Operator → Mapping launch: launches (evidence: `README.md`)
- Mapping launch → Segmentation node: starts (evidence: `README.md`)
- Mapping launch → OctoMap server: starts (evidence: `README.md`)
- Mapping launch → RViz: starts (evidence: `README.md`)
- RGB camera → Segmentation node: provides images (evidence: `README.md`)
- LiDAR point cloud → Segmentation node: provides cloud (evidence: `README.md`)
- TF transforms → Segmentation node: provides transforms (evidence: `README.md`)
- Segmentation node → ONNX models: runs inference (evidence: `README.md`)
- Segmentation node → OctoMap server: publishes cloud (evidence: `README.md`)
- OctoMap server → RViz: visualizes map (evidence: `README.md`)
- Segmentation node → Node state and interfaces: uses interfaces (evidence: `segmentation_node/src/segmentation_node.cpp`)
- Operator → Annotation web app: submits prompt
- Annotation web app → Inference server: posts prediction (evidence: `utils/segmentation_with_sam/app.py`)
- Inference server → Text-guided segmentation: predicts with (evidence: `utils/segmentation_with_sam/lang_sam/server.py`)
- Text-guided segmentation → GroundingDINO: detects boxes (evidence: `utils/segmentation_with_sam/lang_sam/lang_sam.py`)
- Text-guided segmentation → SAM masks: predicts masks (evidence: `utils/segmentation_with_sam/lang_sam/lang_sam.py`)
- Training mask generator → Text-guided segmentation: generates annotations (evidence: `utils/segmentation_with_sam/generate_masks_for_training.py`)
- Batch mask generator → Text-guided segmentation: generates annotations (evidence: `utils/segmentation_with_sam/generate_masks_for_training_with_batch_queries.py`)