How scinter-prod/sonar works
Project Type & Purpose
• SONAR is an open-source Python toolkit/library, not a web app or service.
• It provides a modular API for multilingual, multimodal sentence embeddings and related tasks:
• Text ↔ embedding (encoding/decoding, translation)
• Speech ↔ embedding, speech-to-text
• Similarity scoring (BLASER), toxicity classification (MuTox)
• It integrates tightly with PyTorch, torchaudio, Fairseq2, and (optionally) HuggingFace pipelines.
File Structure & Architectural Patterns
• Top-Level:
• .github, CI/CD, docs, licenses
• examples/ (demo notebooks)
• materials/ (static images)
• huggingface_pipelines/ (optional HF wrappers)
• sonar/
– cards/ ← YAML metadata for each model
– inference_pipelines/ ← user-facing pipelines: TextToEmbedding, EmbeddingToText, TextToText, SpeechToEmbedding, SpeechToText
– models/ ← factory/loader/handler/model abstractions per model family (text_sonar, sonar_speech, mutox, blaser, laser2_text)
– nn/ ← core neural building blocks (encoders, decoders, pools)
• tests/ (unit & integration)
• pyproject.toml, requirements*.txt
• Patterns:
• Factory/Handler/Loader under sonar/models → dynamic model instantiation from cards
• Clear separation of concerns: metadata (cards), core NN (nn/), model definitions (models/), user API (inference_pipelines/)
• Extensibility: drop new YAML cards + implement handler to onboard new models
Core Components & Relationships
a. Components:
1. User API – Python classes and CLI in sonar/inference_pipelines (and huggingface_pipelines)
2. Preprocessing – tokenizers (SentencePiece) for text, feature extractors for audio
3. Model Loader/Factory – reads YAML in sonar/cards to locate weights, instantiates PyTorch modules
4. Neural Models – encoder, decoder, conditional_decoder_model, encoder_pooler in sonar/nn + per-model code under sonar/models/*
5. Decoding & Generation – BeamSearchSeq2SeqGenerator or SamplingSeq2SeqGenerator (Fairseq2)
6. Post-processing – similarity scoring (BLASER), toxicity logits (MuTox)
7. External Dependencies – Fairseq2 model hub, PyTorch/torchaudio, HuggingFace transformers/datasets (optional)
8. Storage – local cache ($TORCH_HOME/hub), downloaded weights
b. Data Flow:
Raw input (text or audio)
→ Pipeline (e.g., TextToEmbeddingModelPipeline)
→ Preprocessor (tokenizer/feature extractor)
→ Model Loader/Factory (reads corresponding card)
→ Encoder forward pass → embeddings
→ [optional] Decoder + search → generated text
→ [optional] Scoring head → similarity or toxicity scores
→ Return to user