# lightonai/pylate architecture

> Architecture diagram of the GitHub repository lightonai/pylate, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-05-11). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/lightonai/pylate
- Repository: https://github.com/lightonai/pylate

## Overview

##### Project Type and Purpose
- PyLate is a Python library (not a monolithic app) that wraps and extends Hugging Face/Sentence Transformers for:
– Late-interaction (ColBERT) model training (contrastive, distillation)
– Document retrieval (Voyager index)
– Reranking
– Optional server/API for remote inference
- It provides modules for model definition, loss functions, evaluation, indexing, retrieval, and utilities.

##### File-Structure Analysis
- Top-level:
– pylate/ : core Python package
– docs/ : MkDocs-based documentation
– examples/ : usage scripts
– tests/ : unit tests
– .github/, CI workflows, linter config
- pylate/ subpackages:
– models/: ColBERT model wrapper
– losses/: Contrastive & distillation losses
– evaluation/: BEIR, ColBERT evaluators
– indexes/: Voyager index interface
– retrieve/: ColBERT retrieval logic
– rank/: reranking utilities
– hf_hub/: helpers for Hugging Face model cards
– utils/: data collators, KD preprocessing, batching, distributed logic
– server/: FastAPI-based inference server

##### Key README Insights
- Leverages SentenceTransformerTrainer from sentence-transformers
- Supports single/multi-GPU, fp16/bf16
- Integrates with HF Datasets for both contrastive and distillation pipelines
- Provides high-level APIs:
– models.ColBERT
– losses.{Contrastive, Distillation}
– evaluation.{ColBERTTripletEvaluator, …}
– indexes.Voyager
– retrieve.ColBERT
– rank.rerank
– server.server (FastAPI)

##### Main Components & Relationships
A. Data Sources
– HF Datasets (triplet streams, KD datasets)
– Custom datasets (user code)
B. Data Processing Layer (utils)
– KDProcessing, ColBERTCollator, distributed, iter_batch
– Feeds batches to Trainer
C. Training Pipeline
– SentenceTransformerTrainer (Hugging Face)
– Modules: models.ColBERT ←→ losses.{Contrastive, Distillation} ←→ evaluation
– Outputs: fine-tuned model artifacts
D. Indexing & Retrieval
– indexes.Voyager: builds & queries index
– retrieve.ColBERT: uses model.encode(queries, docs) + Voyager search
E. Reranking
– rank.rerank: reorders doc lists using ColBERT scores
F. Serving Layer (Optional)
– server.server (FastAPI) exposing endpoints:
- /encode_query
- /encode_document
- /fetch_topk
G. External Services & Libraries
– Hugging Face Transformers & Datasets
– Sentence-Transformers Trainer
– Voyager index storage

##### Architectural Patterns & Principles
- Layered modular design:
– Separation of concerns between model definition, training, indexing, retrieval.
- Plug-in losses and evaluators
- Clear API boundaries: user script ↔ library modules ↔ external dependencies
- Support for distributed and mixed precision (fp16, bf16)

## Diagram (Mermaid)

```mermaid
graph TD
    subgraph "Data Sources"
        D1["HF Datasets"]:::external
        D2["Custom Datasets"]:::external
    end

    subgraph "Data Processing Layer"
        U1["ColBERTCollator"]:::utils
        U2["KDProcessing"]:::utils
        U3["iter_batch"]:::utils
        U4["distributed"]:::utils
        U5["tensor conversion"]:::utils
        U6["multi-process"]:::utils
        U7["huggingface_models helpers"]:::utils
    end

    subgraph "Training Pipeline"
        STT["SentenceTransformerTrainer"]:::external
        M1["ColBERT Model"]:::models
        M2["Dense Model"]:::models
        L1["Contrastive Loss"]:::losses
        L2["Distillation Loss"]:::losses
        L3["CachedContrastive Loss"]:::losses
        E1["ColBERT Triplet Evaluator"]:::evaluation
        E2["ColBERT Distillation Evaluator"]:::evaluation
        E3["NanoBEIR Evaluator"]:::evaluation
        E4["PyLate IR Evaluator"]:::evaluation
        E5["BEIR Utilities"]:::evaluation
        E6["Custom Dataset Loader"]:::evaluation
        ModelArtifacts["Fine-tuned Model"]:::data
    end

    subgraph "Indexing & Retrieval"
        I0["Index Base Interface"]:::indexing
        I1["Voyager Index"]:::indexing
        Rv["ColBERT Retriever"]:::retrieve
        QueryEmb["Query Embeddings"]:::data
        DocEmb["Doc Embeddings"]:::data
        TopK["Top-K Results"]:::data
    end

    subgraph "Reranking"
        RR["Re-ranker"]:::rank
        FinalRank["Final Rankings"]:::data
    end

    subgraph "Serving"
        Srv["FastAPI Server"]:::server
        S1["/encode_query"]:::server
        S2["/encode_document"]:::server
        S3["/fetch_topk"]:::server
    end

    subgraph "External Dependencies"
        HFHub["Hugging Face Hub"]:::external
        HFDS["HF Datasets"]:::external
        STLib["Sentence-Transformers"]:::external
        VoyagerStore["Voyager Storage"]:::external
    end

    D1 --> U1
    D2 --> U1
    U1 --> STT
    U2 --> STT
    U3 --> STT
    U4 --> STT
    U5 --> STT
    U6 --> STT
    U7 --> STT

    STT --> M1
    STT --> M2
    M1 --> L1
    M1 --> L2
    M1 --> L3
    STT --> E1
    STT --> E2
    STT --> E3
    STT --> E4
    STT --> E5
    STT --> E6
    STT --> ModelArtifacts

    ModelArtifacts --> I1
    ModelArtifacts --> Rv
    I0 --> I1
    I1 --> Rv

    Rv --> QueryEmb
    Rv --> DocEmb
    QueryEmb --> I1
    I1 --> TopK

    TopK --> RR
    ModelArtifacts --> RR
    RR --> FinalRank

    Srv --> S1
    Srv --> S2
    Srv --> S3
    S1 --> Rv
    S2 --> I1
    S3 --> TopK

    HFHub --> HFDS
    STLib --> STT
    I1 --> VoyagerStore

    click M1 "https://github.com/lightonai/pylate/blob/main/pylate/models/colbert.py"
    click M2 "https://github.com/lightonai/pylate/blob/main/pylate/models/Dense.py"
    click L1 "https://github.com/lightonai/pylate/blob/main/pylate/losses/contrastive.py"
    click L2 "https://github.com/lightonai/pylate/blob/main/pylate/losses/distillation.py"
    click L3 "https://github.com/lightonai/pylate/blob/main/pylate/losses/cached_contrastive.py"
    click E1 "https://github.com/lightonai/pylate/blob/main/pylate/evaluation/colbert_triplet.py"
    click E2 "https://github.com/lightonai/pylate/blob/main/pylate/evaluation/colbert_distillation.py"
    click E3 "https://github.com/lightonai/pylate/blob/main/pylate/evaluation/nano_beir_evaluator.py"
    click E4 "https://github.com/lightonai/pylate/blob/main/pylate/evaluation/pylate_information_retrieval_evaluator.py"
    click E5 "https://github.com/lightonai/pylate/blob/main/pylate/evaluation/beir.py"
    click E6 "https://github.com/lightonai/pylate/blob/main/pylate/evaluation/custom_dataset.py"
    click I1 "https://github.com/lightonai/pylate/blob/main/pylate/indexes/voyager.py"
    click I0 "https://github.com/lightonai/pylate/blob/main/pylate/indexes/base.py"
    click Rv "https://github.com/lightonai/pylate/blob/main/pylate/retrieve/colbert.py"
    click RR "https://github.com/lightonai/pylate/blob/main/pylate/rank/rank.py"
    click Srv "https://github.com/lightonai/pylate/blob/main/pylate/server/server.py"
    click HFHub "https://github.com/lightonai/pylate/blob/main/pylate/hf_hub/model_card.py"
    click U1 "https://github.com/lightonai/pylate/blob/main/pylate/utils/collator.py"
    click U2 "https://github.com/lightonai/pylate/blob/main/pylate/utils/processing.py"
    click U3 "https://github.com/lightonai/pylate/blob/main/pylate/utils/iter_batch.py"
    click U4 "https://github.com/lightonai/pylate/blob/main/pylate/utils/distributed.py"
    click U5 "https://github.com/lightonai/pylate/blob/main/pylate/utils/tensor.py"
    click U6 "https://github.com/lightonai/pylate/blob/main/pylate/utils/multi_process.py"
    click U7 "https://github.com/lightonai/pylate/blob/main/pylate/utils/huggingface_models.py"

    classDef models fill:#D6EAF8,stroke:#1B4F72
    classDef losses fill:#FADBD8,stroke:#922B21
    classDef evaluation fill:#EBDEF0,stroke:#6C3483
    classDef utils fill:#D5F5E3,stroke:#1D8348
    classDef external fill:#FCF3CF,stroke:#B7950B
    classDef data fill:#D4EFDF,stroke:#1E8449
    classDef indexing fill:#AED6F1,stroke:#21618C
    classDef retrieve fill:#A9DFBF,stroke:#196F3D
    classDef rank fill:#F9E79F,stroke:#B9770E
    classDef server fill:#E8DAEF,stroke:#7D3C98
```
