# scinter-prod/sonar architecture

> Architecture diagram of the GitHub repository scinter-prod/sonar, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-05-05). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/scinter-prod/sonar
- Repository: https://github.com/scinter-prod/sonar

## Overview

##### Project Type & Purpose
- SONAR is an open-source Python toolkit/library, not a web app or service.
- It provides a modular API for multilingual, multimodal sentence embeddings and related tasks:
- Text ↔ embedding (encoding/decoding, translation)
- Speech ↔ embedding, speech-to-text
- Similarity scoring (BLASER), toxicity classification (MuTox)
- It integrates tightly with PyTorch, torchaudio, Fairseq2, and (optionally) HuggingFace pipelines.

##### File Structure & Architectural Patterns
- Top-Level:
- .github, CI/CD, docs, licenses
- examples/ (demo notebooks)
- materials/ (static images)
- huggingface_pipelines/ (optional HF wrappers)
- sonar/
– cards/            ← YAML metadata for each model
– inference_pipelines/ ← user-facing pipelines: TextToEmbedding, EmbeddingToText, TextToText, SpeechToEmbedding, SpeechToText
– models/           ← factory/loader/handler/model abstractions per model family (text_sonar, sonar_speech, mutox, blaser, laser2_text)
– nn/               ← core neural building blocks (encoders, decoders, pools)
- tests/ (unit & integration)
- pyproject.toml, requirements*.txt
- Patterns:
- Factory/Handler/Loader under sonar/models → dynamic model instantiation from cards
- Clear separation of concerns: metadata (cards), core NN (nn/), model definitions (models/), user API (inference_pipelines/)
- Extensibility: drop new YAML cards + implement handler to onboard new models

##### Core Components & Relationships
a. Components:
1. User API – Python classes and CLI in sonar/inference_pipelines (and huggingface_pipelines)
2. Preprocessing – tokenizers (SentencePiece) for text, feature extractors for audio
3. Model Loader/Factory – reads YAML in sonar/cards to locate weights, instantiates PyTorch modules
4. Neural Models – encoder, decoder, conditional_decoder_model, encoder_pooler in sonar/nn + per-model code under sonar/models/*
5. Decoding & Generation – BeamSearchSeq2SeqGenerator or SamplingSeq2SeqGenerator (Fairseq2)
6. Post-processing – similarity scoring (BLASER), toxicity logits (MuTox)
7. External Dependencies – Fairseq2 model hub, PyTorch/torchaudio, HuggingFace transformers/datasets (optional)
8. Storage – local cache ($TORCH_HOME/hub), downloaded weights

b. Data Flow:
Raw input (text or audio)
→ Pipeline (e.g., TextToEmbeddingModelPipeline)
→ Preprocessor (tokenizer/feature extractor)
→ Model Loader/Factory (reads corresponding card)
→ Encoder forward pass → embeddings
→ [optional] Decoder + search → generated text
→ [optional] Scoring head → similarity or toxicity scores
→ Return to user

## Diagram (Mermaid)

```mermaid
flowchart TB
    %% Data Input
    RawInput["Raw Input\n(Text/Audio)"]:::pipeline --> PipelineLayer

    %% User Layer
    subgraph "User Layer"
        direction TB
        Notebook["Notebooks & Tests"]:::userblue
        CLI["CLI Scripts (CLI)"]:::userblue
    end
    Notebook --> PipelineLayer
    CLI --> PipelineLayer

    %% Pipeline Layer
    subgraph "Pipeline Layer" 
        direction TB
        PipelineLayer["Sonar Inference Pipelines\n(sonar/inference_pipelines/)"]:::pipeline
        HFExt["HuggingFace Pipelines\n(huggingface_pipelines/)"]:::pipeline
    end
    PipelineLayer --> CardLayer
    PipelineLayer --> ModelFactory
    HFExt --> ModelFactory

    %% Model Management Layer
    subgraph "Model Management" 
        direction TB
        CardLayer["YAML Cards\n(sonar/cards/*.yaml)"]:::metadata
        ModelFactory["Factory/Loader/Handler\n(sonar/models/*)"]:::metadata
    end
    ModelFactory --> CoreLayer
    CardLayer -.-> ModelFactory

    %% Core NN & Models
    subgraph "Core NN & Model Definitions"
        direction TB
        NN["Neural Building Blocks\n(sonar/nn/)"]:::core
        ModelDef["Model Definitions\n(sonar/models/*/model.py)"]:::core
    end
    CoreLayer --> NN
    CoreLayer --> ModelDef

    %% External Dependencies & Storage
    subgraph "External & Storage"
        direction TB
        FairseqHub["Fairseq2 Model Hub"]:::external
        LocalCache["Local Cache\n($TORCH_HOME/hub)"]:::external
        TorchLibs["PyTorch & Torchaudio"]:::external
    end
    ModelFactory -->|reads weights| LocalCache
    ModelFactory -->|fetch models| FairseqHub
    NN --> TorchLibs
    PipelineLayer --> TorchLibs

    %% Click Events
    click PipelineLayer "https://github.com/scinter-prod/sonar/tree/main/sonar/inference_pipelines/"
    click HFExt "https://github.com/scinter-prod/sonar/tree/main/huggingface_pipelines/"
    click CardLayer "https://github.com/scinter-prod/sonar/tree/main/sonar/cards/"
    click ModelFactory "https://github.com/scinter-prod/sonar/blob/main/sonar/models/*/factory.py"
    click NN "https://github.com/scinter-prod/sonar/blob/main/sonar/nn/conditional_decoder_model.py"
    click ModelDef "https://github.com/scinter-prod/sonar/blob/main/sonar/models/sonar_text/model.py"

    %% Styles
    classDef userblue fill:#D6EAF8,stroke:#21618C,color:#154360;
    classDef pipeline fill:#AED6F1,stroke:#1B4F72,color:#1A5276;
    classDef metadata fill:#FCF3CF,stroke:#B7950B,color:#7D6608;
    classDef core fill:#D5F5E3,stroke:#196F3D,color:#145A32;
    classDef external fill:#E5E7E9,stroke:#566573,color:#424949;
```
