# anush008/fastembed-rs architecture

> Architecture diagram of the GitHub repository anush008/fastembed-rs, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-02-14). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/anush008/fastembed-rs
- Repository: https://github.com/anush008/fastembed-rs

## Overview

##### Project Type and Purpose
- Project Type: A Rust-based library designed to provide fast embedding generation and candidate reranking functionality using ONNX models.
- Purpose: It abstracts the underlying complexity of model selection, inference (via ONNX runtime via the “ort” crate), fast encoding (via Hugging Face’s tokenizers), and parallel processing (via Rayon), making it easy for users to compute embeddings and rerank results for tasks such as semantic search.

##### File Structure
- Top-level Files and Directories:
- Cargo.toml, LICENSE, README.md: These are standard for a Rust crate.
- .github/ – contains workflows for CI/CD and releases.
- src/ – contains the library’s implementation.
- tests/ – contains integration/unit tests and sample assets.
- Inside src/:
- lib.rs: The main library entry point that wires together the public API.
- common.rs: Likely holds shared utilities or types.
- image_embedding, text_embedding, sparse_text_embedding, reranking: Each provides initialization, inference implementation, and helper functions for specific models.
- models/ – defines the model information, quantization settings, and possibly the customization of different embedding models.
- output/ – handles formatting or choices regarding embedding output.
- pooling.rs – might encapsulate operations for pooling or aggregating output embeddings.
- The structure indicates modular design with each embedding type and reranking functionality logically separated, suggesting that the internal design uses component segregation to ensure maintainability and flexibility.

##### README for Additional Insights
- The README provides clear usage examples and shows that clients initialize a model (either text, image, or reranking) with specific options.
- External dependencies include:
- Pykeio’s ONNX runtime for inference.
- Hugging Face tokenizers for encoding.
- Rayon for parallelism on batch processing.
- There is an emphasis on ease of use “synchronous usage with no dependency on Tokio,” reinforcing that the operational flow is designed to run synchronously.

a. Main Components:
- API Layer (Public Interface):
- Initialization functions for TextEmbedding, ImageEmbedding, SparseTextEmbedding, and TextRerank.
- Client code examples in the README indicate how these components are used.
- Model Management:
- The models/ directory contains configurations and model-specific logic.
- Inference Modules:
- image_embedding, text_embedding, sparse_text_embedding: Responsible for the ONNX model interactions.
- reranking: Contains logic to rerank candidate results.
- Utilities & Shared Logic:
- common.rs, pooling.rs, and parts of output/ provide cross-cutting utilities and assist in data formatting.
- Testing:
- tests/ directory demonstrates the expected behavior of the components and can be used to verify the design.
- External Service Integration:
- Dependency on “ort” for ONNX runtime, “tokenizers” for fast processing, and “rayon” for parallel execution.

b. Relationships and Interactions:
- The public API (in lib.rs) orchestrates calls into specific modules (text_embedding, image_embedding, etc.).
- Each module retrieves model-specific settings from the models/ directory.
- Data flows from user inputs to the initialization layers, then through the respective embedding modules which internally call the ONNX runtime library, and finally through the output and pooling modules to format and possibly post-process the embeddings.
- The reranking component may take outputs from one or several other modules and apply an additional processing layer to reorder results.

c. Architectural Patterns:
- Modularization: Clear separation into modules based on embedding type and functional responsibilities.
- Dependency Injection: Initialization options (e.g., InitOptions, ImageInitOptions, etc.) allow for dynamic configuration of the underlying models.
- Abstraction: The user-facing interfaces abstract away the lower-level complexities of interacting with the ONNX runtime.
- Parallel Processing: Use of Rayon indicates that batch processing is designed for parallelism.

d. Relevant Technologies/Frameworks:
- Rust as the implementation language.
- ONNX Runtime via the “ort” crate.
- Hugging Face Tokenizers.
- Rayon for parallelized tasks.
- GitHub Actions (as seen in the .github/workflows) for CI/CD.

##### Instructing the Principal Software Engineer to Include Specific Elements
- For example, label nodes “Text Embedding Module,” “Image Embedding Module,” “Sparse Embedding Module,” “Reranking Module,” “Model Configurations,” “Output Formatter,” etc.
- Public interfaces vs. internal utility modules.
- External services (ONNX, Tokenizers, Rayon) vs. internal components.

This level of detailed separation will accurately capture the essential architectural elements and ensure clarity, comprehensiveness, and ease of understanding, aligning with the specific nature, components, and technical dependencies of FastEmbed‑rs.

## Diagram (Mermaid)

```mermaid
flowchart TD
    subgraph "FastEmbed‑rs Library"
        API["API Layer (lib.rs)"]
        subgraph "Core Modules"
            TE["Text Embedding Module"]
            IE["Image Embedding Module"]
            STE["Sparse Text Embedding Module"]
            RR["Reranking Module"]
            MM["Model Management"]
            OF["Output Formatter"]
            US["Utilities & Shared Logic"]
        end
    end

    subgraph "External Dependencies"
        OR["ONNX Runtime"]
        TK["Tokenizers"]
        RAY["Rayon"]
    end

    %% API calls to Inference Modules
    API -->|"calls"| TE
    API -->|"calls"| IE
    API -->|"calls"| STE
    API -->|"calls"| RR

    %% Inference Modules fetching model configuration
    TE -->|"fetches"| MM
    IE -->|"fetches"| MM
    STE -->|"fetches"| MM
    RR -->|"fetches"| MM

    %% Inference Modules output to Output Formatter
    TE -->|"outputs"| OF
    IE -->|"outputs"| OF
    STE -->|"outputs"| OF
    RR -->|"outputs"| OF

    %% Inference Modules use Utilities & Shared Logic
    TE ---|"utilizes"| US
    IE ---|"utilizes"| US
    STE ---|"utilizes"| US
    RR ---|"utilizes"| US

    %% Inference Modules use External Dependencies
    TE -->|"uses"| OR
    TE -->|"uses"| TK
    TE -->|"uses"| RAY

    IE -->|"uses"| OR
    IE -->|"uses"| TK
    IE -->|"uses"| RAY

    STE -->|"uses"| OR
    STE -->|"uses"| TK
    STE -->|"uses"| RAY

    RR -->|"uses"| OR
    RR -->|"uses"| TK
    RR -->|"uses"| RAY

    %% Click events for component mapping
    click API "https://github.com/anush008/fastembed-rs/blob/main/src/lib.rs"
    click TE "https://github.com/anush008/fastembed-rs/tree/main/src/text_embedding"
    click IE "https://github.com/anush008/fastembed-rs/tree/main/src/image_embedding"
    click STE "https://github.com/anush008/fastembed-rs/tree/main/src/sparse_text_embedding"
    click RR "https://github.com/anush008/fastembed-rs/tree/main/src/reranking"
    click MM "https://github.com/anush008/fastembed-rs/tree/main/src/models"
    click OF "https://github.com/anush008/fastembed-rs/tree/main/src/output"
    click US "https://github.com/anush008/fastembed-rs/blob/main/src/common.rs"
    click US "https://github.com/anush008/fastembed-rs/blob/main/src/pooling.rs"

    %% Styles
    classDef api fill:#DDEEFF,stroke:#6B8E23,stroke-width:2px;
    classDef module fill:#FFEECC,stroke:#FF9900,stroke-width:2px;
    classDef external fill:#EEFFEE,stroke:#008000,stroke-width:2px;
    classDef utility fill:#F0E68C,stroke:#B8860B,stroke-width:2px;

    class API api;
    class TE,IE,STE,RR,MM,OF module;
    class US utility;
    class OR,TK,RAY external;
```
