# intellabs/fastrag architecture

> Architecture diagram of the GitHub repository intellabs/fastrag, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-05-30). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/intellabs/fastrag
- Repository: https://github.com/intellabs/fastrag

## Overview

##### Project Type and Purpose
- fastRAG is an open-source research framework for building efficient retrieval-augmented generation (RAG) pipelines.
- It combines modular IR components (retrievers, rankers, document stores) with generative LLM backends (FiD, REPLUG, Gaudi, OpenVINO, Llama-CPP).
- Designed for extensibility (Haystack compatibility), optimized inference (Intel IPEX, Gaudi, ONNX), and multiple storage backends (Elasticsearch, Faiss, Qdrant, PLAID).

##### File-Structure Analysis
- Top-level:
- fastrag/           ← core Python package
- config/            ← YAML pipeline/component definitions
- scripts/           ← pipeline generation, indexing, training, quantization
- benchmarks/        ← evaluation scripts for BEIR, KILT
- assets/, examples/ ← sample data & notebooks
- README.md, setup.py, etc.
- Within fastrag/:
- agents/            ← conversational agent orchestrator + memory + tools
- embedders/, retrievers/, rankers/, stores/ ← IR components
- generators/        ← reader/generation backends (FiD, REPLUG, ONNX, Gaudi, Llama-CPP)
- prompt_builders/, prompt_compressors/ ← prompt-engineering modules
- ui/                ← Chainlit web UIs for demos

a. Main Components
- Configuration Layer: YAML files in config/ define pipeline graphs.
- Pipeline Builder: scripts/generate_pipeline.py + Haystack-compatible factory methods read configs.
- IR Tier:
- Document Stores (Elasticsearch, Faiss, Qdrant, PLAID)
- Embedders & Retrievers (SBERT, BM25, ColBERT-v2)
- Rankers (cross-encoders, sparse models)
- Generation Tier:
- Reader / LLM wrappers (Fusion-in-Decoder, REPLUG)
- Hardware-accelerated backends (IPEX, Optimum-Intel, ONNX, Gaudi)
- Alternative backends (OpenVINO, Llama-CPP)
- Agent & Orchestration:
- AgentStep, tools & memory (fastrag/agents)
- Prompt builders & compressors
- User Interfaces:
- Chainlit UI modules (chat, multi-modal, pipelines)
- CLI / Python API
- Auxiliary:
- Scripts for indexing, training, quantization, benchmarks

b. Relationships and Data Flows
1. User → Config Loader → Pipeline Builder
2. Pipeline Builder instantiates:
- DocumentStore ← data loading/indexing scripts
- Retriever → initial document embeddings or BM25 scores
- Ranker → refines top documents
- Generator → LLM backend produces answer
3. Agent orchestrates retrieval/generation sequence, manages conversation memory, calls tools if needed.
4. UI or Python API receives final output; benchmarks and scripts invoke pipelines programmatically.

c. Architectural Patterns
- Plugin / Factory: component classes loaded based on YAML configs
- Pipeline / DAG: sequential IR → ranking → generation stages
- Separation of Concerns: distinct modules for storage, retrieval, ranking, generation, UI
- Extensibility Points: custom embedders, retrievers, generators via clear module hierarchy

d. Key Technologies
- Haystack v2+ for pipeline abstractions
- HuggingFace Transformers / Optimum Intel / Optimum-Habana
- Intel IPEX, ONNX Runtime, OpenVINO, Gaudi accelerators, Llama-CPP
- Elasticsearch, Faiss (CPU/GPU), Qdrant, PLAID

##### Essential Elements to Capture
- Clear, descriptive labels for every component and storage backend.

## Diagram (Mermaid)

```mermaid
graph TD
    %% Configuration Layer
    subgraph "Configuration Layer"
        Config["YAML Config Files"]:::config
    end

    %% Pipeline Builder
    PipelineBuilder["Pipeline Builder"]:::core
    click PipelineBuilder "https://github.com/intellabs/fastrag/blob/main/scripts/generate_pipeline.py"

    Config -->|"reads configs"| PipelineBuilder

    %% IR Tier
    subgraph "IR Tier"
        subgraph "Document Stores"
            ES["Elasticsearch"]:::store
            Faiss["Faiss"]:::store
            Qdrant["Qdrant"]:::store
            PLAID["PLAID"]:::store
            click ES "https://github.com/intellabs/fastrag/blob/main/fastrag/stores/elastic.py"
            click Faiss "https://github.com/intellabs/fastrag/blob/main/fastrag/stores/faiss.py"
            click Qdrant "https://github.com/intellabs/fastrag/blob/main/fastrag/stores/qdrant.py"
            click PLAID "https://github.com/intellabs/fastrag/blob/main/fastrag/stores/plaid.py"
        end

        subgraph "Embedders & Retrievers"
            IPEXEmbed["IPEX Embedder"]:::core
            SBERT["SBERT Embedder"]:::core
            BM25["BM25 Retriever"]:::core
            ColBERT["ColBERT-v2 Retriever"]:::core
            click IPEXEmbed "https://github.com/intellabs/fastrag/blob/main/fastrag/embedders/ipex_embedder.py"
            click SBERT "https://github.com/intellabs/fastrag/blob/main/fastrag/embedders/__init__.py"
            click BM25 "https://github.com/intellabs/fastrag/blob/main/fastrag/retrievers/elastic.py"
            click ColBERT "https://github.com/intellabs/fastrag/blob/main/fastrag/retrievers/colbert.py"
        end

        subgraph "Rankers"
            BiEnc["Bi-Encoder Ranker"]:::core
            CrossEnc["Cross-Encoder Ranker"]:::core
            IPEXRank["IPEX Bi-Encoder Ranker"]:::core
            click BiEnc "https://github.com/intellabs/fastrag/blob/main/fastrag/rankers/bi_encoder_ranker.py"
            click CrossEnc "https://github.com/intellabs/fastrag/blob/main/fastrag/rankers/colbert.py"
            click IPEXRank "https://github.com/intellabs/fastrag/blob/main/fastrag/rankers/ipex_bi_encoder_ranker.py"
        end
    end

    PipelineBuilder -->|"instantiate"| ES
    PipelineBuilder -->|"instantiate"| Faiss
    PipelineBuilder -->|"instantiate"| Qdrant
    PipelineBuilder -->|"instantiate"| PLAID

    ES -->|"store docs"| BM25
    Faiss -->|"store docs"| ColBERT
    Qdrant -->|"store docs"| IPEXEmbed
    PLAID -->|"store docs"| SBERT

    BM25 -->|"initial scores"| BiEnc
    ColBERT -->|"embeddings"| CrossEnc
    IPEXEmbed -->|"embeddings"| IPEXRank
    SBERT -->|"embeddings"| BiEnc

    BiEnc --> RankOut["Ranked Docs"]:::core
    CrossEnc --> RankOut
    IPEXRank --> RankOut

    %% Generation Tier
    subgraph "Generation Tier"
        FiD["Fusion-in-Decoder"]:::core
        Replug["REPLUG"]:::core
        IPEXBack["Intel IPEX Backend"]:::backend
        ONNX["ONNX Runtime"]:::backend
        Gaudi["Gaudi Backend"]:::backend
        OpenVINO["OpenVINO"]:::backend
        LLAMA["Llama-CPP"]:::backend
        click FiD "https://github.com/intellabs/fastrag/blob/main/fastrag/generators/fid.py"
        click Replug "https://github.com/intellabs/fastrag/blob/main/fastrag/generators/replug.py"
        click IPEXBack "https://github.com/intellabs/fastrag/blob/main/fastrag/generators/ipex.py"
        click ONNX "https://github.com/intellabs/fastrag/blob/main/fastrag/generators/ort.py"
        click Gaudi "https://github.com/intellabs/fastrag/blob/main/fastrag/generators/gaudi.py"
        click OpenVINO "https://github.com/intellabs/fastrag/blob/main/fastrag/generators/openvino.py"
        click LLAMA "https://github.com/intellabs/fastrag/blob/main/fastrag/generators/llava.py"
    end

    RankOut -->|"prompt -> docs"| FiD
    RankOut -->|"prompt -> docs"| Replug
    RankOut -->|"prompt -> docs"| IPEXBack
    RankOut -->|"prompt -> docs"| ONNX
    RankOut -->|"prompt -> docs"| Gaudi
    RankOut -->|"prompt -> docs"| OpenVINO
    RankOut -->|"prompt -> docs"| LLAMA

    %% Prompt Engineering
    subgraph "Prompt Engineering"
        PB["Prompt Builders"]:::core
        PC1["Prompt Compressor (lingua)"]:::core
        PC2["Prompt Compressor (OpenVINO)"]:::core
        click PB "https://github.com/intellabs/fastrag/blob/main/fastrag/prompt_builders/multi_modal_prompt_builder.py"
        click PC1 "https://github.com/intellabs/fastrag/blob/main/fastrag/prompt_compressors/llm_lingua.py"
        click PC2 "https://github.com/intellabs/fastrag/blob/main/fastrag/prompt_compressors/ov_llm_lingua.py"
    end

    FiD -->|"prompt"| PB
    Replug -->|"prompt"| PB
    PB -->|"compressed prompt"| PC1
    PC1 -->|"final prompt"| PC2

    %% Agent & Orchestration
    subgraph "Agent & Orchestration"
        Agent["Agent"]:::orchestration
        AgentStep["AgentStep"]:::orchestration
        CreateAgent["Agent Factory"]:::orchestration
        Memory["Conversation Memory"]:::orchestration
        Tools["Tools"]:::orchestration
        Utils["Agent Utils"]:::orchestration
        click Agent "https://github.com/intellabs/fastrag/blob/main/fastrag/agents/base.py"
        click AgentStep "https://github.com/intellabs/fastrag/blob/main/fastrag/agents/agent_step.py"
        click CreateAgent "https://github.com/intellabs/fastrag/blob/main/fastrag/agents/create_agent.py"
        click Memory "https://github.com/intellabs/fastrag/blob/main/fastrag/agents/memory/conversation_memory.py"
        click Tools "https://github.com/intellabs/fastrag/blob/main/fastrag/agents/tools/tools.py"
        click Utils "https://github.com/intellabs/fastrag/blob/main/fastrag/agents/utils.py"
    end

    Agent -->|"manages"| AgentStep
    Agent -->|"uses"| Memory
    Agent -->|"invokes"| Tools
    Agent -->|"applies"| Utils
    AgentStep -->|"calls"| FiD
    AgentStep -->|"calls"| Replug

    %% UI & API
    subgraph "UI & API"
        ChainlitUI["Chainlit UI"]:::ui
        CLI["CLI / Python API"]:::ui
        click ChainlitUI "https://github.com/intellabs/fastrag/blob/main/fastrag/ui/chainlit_pipeline.py"
        click ChainlitCallback "https://github.com/intellabs/fastrag/blob/main/fastrag/ui/chainlit_agent_fastrag_callback.py"
        click MMUI "https://github.com/intellabs/fastrag/blob/main/fastrag/ui/chainlit_multi_modal.py"
        click MMAgentUI "https://github.com/intellabs/fastrag/blob/main/fastrag/ui/chainlit_multi_modal_agent.py"
        click NoRAGUI "https://github.com/intellabs/fastrag/blob/main/fastrag/ui/chainlit_no_rag.py"
    end

    FiD -->|"response"| ChainlitUI
    Agent -->|"response"| ChainlitUI
    ChainlitUI -->|"CLI call"| CLI

    %% Auxiliary Scripts
    subgraph "Auxiliary Scripts"
        DataLoad["Data Loading & Indexing"]:::aux
        TrainOpt["Training & Optimization"]:::aux
        Bench["Benchmarks"]:::aux
        click DataLoad "https://github.com/intellabs/fastrag/blob/main/fastrag/data_loaders.py"
        click createElastic "https://github.com/intellabs/fastrag/blob/main/scripts/indexing/create_elastic.py"
        click createFaiss "https://github.com/intellabs/fastrag/blob/main/scripts/indexing/create_faiss.py"
        click createPlaid "https://github.com/intellabs/fastrag/blob/main/scripts/indexing/create_plaid.py"
        click createDense "https://github.com/intellabs/fastrag/blob/main/scripts/indexing/create_dense.py"
        click trainFID "https://github.com/intellabs/fastrag/blob/main/scripts/training/train_fid.py"
        click quantEmbed "https://github.com/intellabs/fastrag/blob/main/scripts/optimizations/embedders/quantize_embedder.py"
        click quantRank "https://github.com/intellabs/fastrag/blob/main/scripts/optimizations/reranker_quantization/quantize_model.py"
        click trainKD "https://github.com/intellabs/fastrag/blob/main/scripts/optimizations/reranker_quantization/train_cross-encoder_kd.py"
        click BEIRms "https://github.com/intellabs/fastrag/blob/main/benchmarks/BEIR/msmarco-bm25-sbert.py"
        click BEIRnq "https://github.com/intellabs/fastrag/blob/main/benchmarks/BEIR/nq-bm25-sbert.py"
        click BEIRmsPlaid "https://github.com/intellabs/fastrag/blob/main/benchmarks/BEIR/msmarco-plaid.py"
        click BEIRnqPlaid "https://github.com/intellabs/fastrag/blob/main/benchmarks/BEIR/nq-plaid.py"
        click KILT1 "https://github.com/intellabs/fastrag/blob/main/benchmarks/KILT/nq-bm25-fid.py"
        click KILT2 "https://github.com/intellabs/fastrag/blob/main/benchmarks/KILT/nq-plaid-fid.py"
    end

    DataLoad -->|"feeds"| PipelineBuilder
    TrainOpt -->|"optimizes"| PipelineBuilder
    Bench -->|"evaluates"| PipelineBuilder

    %% Styles
    classDef core fill:#E3F2FD,stroke:#2196F3,color:#0D47A1
    classDef store fill:#E8F5E9,stroke:#4CAF50,color:#1B5E20
    classDef backend fill:#FFF3E0,stroke:#FF9800,color:#E65100
    classDef orchestration fill:#F3E5F5,stroke:#9C27B0,color:#4A148C
    classDef ui fill:#EDE7F6,stroke:#673AB7,color:#311B92
    classDef aux fill:#ECEFF1,stroke:#607D8B,color:#263238
    classDef config fill:#E1F5FE,stroke:#03A9F4,color:#01579B
```
