# guillaume-be/rust-bert architecture

> Architecture diagram of the GitHub repository guillaume-be/rust-bert, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-03-31). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/guillaume-be/rust-bert
- Repository: https://github.com/guillaume-be/rust-bert

## Overview

##### Main Components
- Core Library (src/lib.rs): This is the heart of the project. It includes:
- Common components (in src/common): Modules for activations, dropout, embeddings, configuration, error handling, and resource management (local, remote, buffer).
- Model implementations (in src/models): Each transformer model (BERT, GPT2, ALBERT, etc.) is organized into its own directory with submodules for attention mechanisms, encoder/decoder layers, embeddings, and other model-specific logic.
- Pipelines (in src/pipelines): These combine the core models with task-specific logic. Different pipelines exist for tasks like question answering, sentiment analysis, translation, summarization, text generation, NER, etc.
- Examples & Benchmarks (examples, benches): These directories provide executable examples and benchmarks to demonstrate usage and measure performance.
- Tests (tests): Contains unit tests for individual model implementations and pipelines.
- Utilities (utils): Scripts and helper modules for operations like converting model weights.

##### External Dependencies & Interactions
- The project uses the tch-rs crate (Rust binding for Libtorch) for model execution using native or GPU-accelerated inference.
- Optionally, it includes ONNX support via bindings from the ort crate.
- It relies on rust-tokenizers for pre-processing and tokenization.
- Additional build and configuration files (Cargo.toml, build.rs) manage the Rust build process and dependencies.

##### Determining Relationships & Data Flow
- At the top level, the user’s code (or the examples) calls one of the ready-to-use pipeline APIs.
- The pipeline modules encapsulate task-specific logic, calling into the model components (from src/models) and leveraging shared common utility functions (from src/common).
- Model components then rely on lower-level tensor operations through tch-rs (or ONNX if the optional feature is enabled), which in turn interacts with the underlying C++ libraries (libtorch or onnxruntime).
- The utilities (such as the conversion script) interface with external pre-trained model weights to prepare or convert them to the required format.

##### Architectural Patterns & Design Principles
- Modularization: There is a clear separation between “common” utilities, model definitions, and task-specific pipelines.
- Layered design: The top-level APIs and pipelines abstract away the low-level details of model execution and pre-processing.
- Extensibility: With separate modules for each transformer model and pipeline, adding a new model architecture or pipeline is straightforward.
- Feature flags: Optional ONNX support is implemented as a feature flag, demonstrating the use of conditional compilation and configuration.

## Diagram (Mermaid)

```mermaid
flowchart TD
    %% User Interaction
    subgraph "User Interaction"
        A["User Code / Examples"]:::external
    end

    %% Core Library Layer
    subgraph "Core Library"
        CL["src/lib.rs"]:::logic
    end

    %% Common Components
    subgraph "Common Components"
        CC["src/common"]:::logic
    end

    %% Transformer Models
    subgraph "Transformer Models"
        TM["src/models"]:::logic
    end

    %% Pipelines Layer
    subgraph "Pipelines"
        P["src/pipelines"]:::logic
    end

    %% Utilities & Examples
    subgraph "Utilities & Examples"
        EX["examples"]:::utility
        BN["benches"]:::utility
        TS["tests"]:::utility
        UT["utils"]:::utility
        CT["Cargo.toml"]:::utility
        BR["build.rs"]:::utility
    end

    %% External Dependencies
    subgraph "External Dependencies"
        TCH["tch-rs/libtorch"]:::external
        ONNX["ONNX Runtime(ort)"]:::external
        RTOK["rust-tokenizers"]:::external
    end

    %% Data Flow Connections
    A -->|"triggers"| P
    A --> EX
    P -->|"calls"| TM
    P -->|"uses"| CC
    P -->|"pre-processes"| RTOK
    TM -->|"utilizes"| TCH
    TM -->|"optionally_uses"| ONNX
    CC -->|"supports"| TM
    UT -->|"assists"| CC
    UT -->|"converts_weights_for"| TM
    CL --> CC
    CL --> TM
    CT -->|"configures"| CL
    BR -->|"configures"| CL
    BN --> TS

    %% Click Events
    click CL "https://github.com/guillaume-be/rust-bert/blob/main/src/lib.rs"
    click CC "https://github.com/guillaume-be/rust-bert/tree/main/src/common"
    click TM "https://github.com/guillaume-be/rust-bert/tree/main/src/models"
    click P "https://github.com/guillaume-be/rust-bert/tree/main/src/pipelines"
    click EX "https://github.com/guillaume-be/rust-bert/tree/main/examples"
    click BN "https://github.com/guillaume-be/rust-bert/tree/main/benches"
    click TS "https://github.com/guillaume-be/rust-bert/tree/main/tests"
    click UT "https://github.com/guillaume-be/rust-bert/tree/main/utils"
    click CT "https://github.com/guillaume-be/rust-bert/blob/main/Cargo.toml"
    click BR "https://github.com/guillaume-be/rust-bert/blob/main/build.rs"

    %% Styles
    classDef logic fill:#AED6F1,stroke:#2471A3,stroke-width:2px;
    classDef utility fill:#ABEBC6,stroke:#28B463,stroke-width:2px;
    classDef external fill:#F9E79F,stroke:#F1C40F,stroke-width:2px;
```
