# ysj9909/soma architecture

> Architecture diagram of the GitHub repository ysj9909/soma, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-10-10). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/ysj9909/soma
- Repository: https://github.com/ysj9909/soma

## Overview

##### Project type and purpose
- This is a research‐grade machine‐learning codebase for domain-generalized semantic segmentation and object detection.
- It builds on open-source toolkits (MMDetection, MMSegmentation) and a Parameter-Efficient Fine-Tuning (PEFT) method called SoMA, applying singular value decomposition to minor components of backbone weights (DINOv2) for robust performance on unseen domains.

##### File structure
- Top-level directories:
- configs/  – base and project-specific configuration files
- DGOD/     – “Domain Generalizable Object Detection” integration with mmdet
- rein/     – “Rein” integration for segmentation (backbones & segmentors)
- projects/ – example project forks (CO_DETR)
- Key patterns:
- Config-driven architecture: configs organize datasets, models, schedules, runtime.
- Modular model code under DGOD/mmdet and rein/models, following registry+builder patterns.
- Training/eval drivers in tools and DGOD/tools.
- Pre-trained weight conversion & merging scripts for inference pipeline.

##### README for additional insights
- Describes SoMA’s purpose: freeze major singular values, adapt minor ones for domain generalization.
- Provides environment setup, data preparation pointers (Rein, Diverse Weather), weight conversion, merging, training and evaluation commands.
- Highlights two tasks: Domain-Generalized Semantic Segmentation (under rein + MMSegmentation) and Domain-Generalized Object Detection (under DGOD + MMDetection).

##### Elements
- Config → Data Loader / Model Builder / Training Engine
- Data Loader → Model → Loss Computation → Optimizer → Checkpoint
- Pre-trained Weights → Convert Script → Converted Backbone
- Converted Backbone + SoMA Checkpoint → Merge Script → Inference Checkpoint
- Inference Checkpoint → Evaluation → Metrics
- Blue for data / dataset modules
- Green for model components
- Orange for training & hooks
- Purple for tools/utilities (conversion, merge)
- Pink for external dependencies

##### Detail without overthinking
- Break out every submodule: configs/_base_, rein/models/backbones, DGOD/mmdet/engine/hooks, etc.
- Both high-level pipelines (seg & det) and deeper building blocks (assigners, samplers, transforms).

## Diagram (Mermaid)

```mermaid
flowchart TB
    %% External Dependencies
    Pretrained["Pre-trained DINOv2 Weights"]:::external
    PyTorch["PyTorch, MMEngine, MMCV"]:::external
    MMFrameworks["MMDetection, MMSegmentation, PEFT, Transformers"]:::external

    %% Configuration Manager
    subgraph "Configuration Manager"
        Configs["configs/"]:::config
        DGOD_Configs["DGOD/configs/"]:::config
    end

    %% Data Preparation
    subgraph "Data Preparation Pipeline"
        ConvertDatasets["tools/convert_datasets/"]:::data
        CityscapesScript["cityscapes.py"]:::data
    end

    %% Model Components
    subgraph "Model Builder"
        Backbone["DINOv2 Backbone"]:::model
        SoMA["SoMA Adapter Modules"]:::model
        subgraph "Segmentation Head"
            Mask2Former1["rein/models/heads/rein_mask2former.py"]:::model
            Mask2Former2["DGOD/mmdet/models/dense_heads/mask2former_head.py"]:::model
        end
        subgraph "Detection Head"
            CoDETR_Config["DGOD/configs/_base_/models/dinov2_codetr.py"]:::model
            CoDETR_Code["DGOD/projects/CO_DETR/codetr/codetr.py"]:::model
        end
    end

    %% Training & Hooks
    subgraph "Training Engine"
        TrainScript["tools/train.py"]:::train
        DistTrain["tools/dist_train.sh"]:::train
        HookSeg["rein/hooks/load_backbone_hook.py"]:::train
        HookDet["DGOD/mmdet/engine/hooks/mean_teacher_hook.py"]:::train
    end

    %% Conversion & Merging
    subgraph "Checkpoint Conversion"
        ConvertBase["tools/convert_models/convert_dinov2.py"]:::tools
        ConvertSora["tools/convert_models/convert_dinov2_sora.py"]:::tools
    end
    MergeScript["tools/merge_soma_weights.py"]:::tools

    %% Evaluation & Visualization
    subgraph "Evaluation Engine"
        TestScript["tools/test.py"]:::eval
        DistTest["tools/dist_test.sh"]:::eval
        Metrics["Evaluation Metrics"]:::eval
    end
    Visualize["tools/visualize.py"]:::tools

    %% Workflow Arrows
    Pretrained -->|provides weights| ConvertBase
    Pretrained -->|provides weights| ConvertSora
    Configs --> ConvertDatasets
    DGOD_Configs --> ConvertDatasets
    Configs --> Backbone
    DGOD_Configs --> CoDETR_Config
    ConvertDatasets -->|prepared datasets| Backbone
    Backbone --> SoMA
    SoMA --> Mask2Former1
    SoMA --> Mask2Former2
    SoMA --> CoDETR_Config
    SoMA --> CoDETR_Code
    Mask2Former1 --> TrainScript
    Mask2Former2 --> TrainScript
    CoDETR_Code --> TrainScript
    TrainScript -->|uses hooks| HookSeg
    TrainScript -->|uses hooks| HookDet
    TrainScript -->|outputs checkpoint| MergeScript
    ConvertBase --> MergeScript
    ConvertSora --> MergeScript
    MergeScript -->|produces inference checkpoint| TestScript
    MergeScript -->|produces inference checkpoint| DistTest
    TestScript --> Metrics
    DistTest --> Metrics
    Metrics --> Visualize

    %% Click Events
    click Configs "https://github.com/ysj9909/soma/tree/main/configs/"
    click DGOD_Configs "https://github.com/ysj9909/soma/tree/main/DGOD/configs/"
    click ConvertDatasets "https://github.com/ysj9909/soma/tree/main/tools/convert_datasets/"
    click CityscapesScript "https://github.com/ysj9909/soma/blob/main/tools/convert_datasets/cityscapes.py"
    click Backbone "https://github.com/ysj9909/soma/blob/main/rein/models/backbones/dino_v2.py"
    click SoMA "https://github.com/ysj9909/soma/blob/main/rein/peft/sora.py"
    click Mask2Former1 "https://github.com/ysj9909/soma/blob/main/rein/models/heads/rein_mask2former.py"
    click Mask2Former2 "https://github.com/ysj9909/soma/blob/main/DGOD/mmdet/models/dense_heads/mask2former_head.py"
    click CoDETR_Config "https://github.com/ysj9909/soma/blob/main/DGOD/configs/_base_/models/dinov2_codetr.py"
    click CoDETR_Code "https://github.com/ysj9909/soma/blob/main/DGOD/projects/CO_DETR/codetr/codetr.py"
    click TrainScript "https://github.com/ysj9909/soma/blob/main/tools/train.py"
    click DistTrain "https://github.com/ysj9909/soma/blob/main/tools/dist_train.sh"
    click TestScript "https://github.com/ysj9909/soma/blob/main/tools/test.py"
    click DistTest "https://github.com/ysj9909/soma/blob/main/tools/dist_test.sh"
    click HookSeg "https://github.com/ysj9909/soma/blob/main/rein/hooks/load_backbone_hook.py"
    click HookDet "https://github.com/ysj9909/soma/blob/main/DGOD/mmdet/engine/hooks/mean_teacher_hook.py"
    click ConvertBase "https://github.com/ysj9909/soma/blob/main/tools/convert_models/convert_dinov2.py"
    click ConvertSora "https://github.com/ysj9909/soma/blob/main/tools/convert_models/convert_dinov2_sora.py"
    click MergeScript "https://github.com/ysj9909/soma/blob/main/tools/merge_soma_weights.py"
    click Visualize "https://github.com/ysj9909/soma/blob/main/tools/visualize.py"

    %% Styles
    classDef external fill:#F9C2CB,stroke:#DB4363,color:#000
    classDef config fill:#C2E0F9,stroke:#3A8EDB,color:#000
    classDef data fill:#C2F9C7,stroke:#3ADB5B,color:#000
    classDef model fill:#D4F9C2,stroke:#7AEC3F,color:#000
    classDef train fill:#F9EAC2,stroke:#DBB543,color:#000
    classDef tools fill:#E2C2F9,stroke:#8E3ADB,color:#000
    classDef eval fill:#F9C2E0,stroke:#DB4391,color:#000
```
