# vectorinstitute/fed-rag architecture

> Architecture diagram of the GitHub repository vectorinstitute/fed-rag, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-03-29). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/vectorinstitute/fed-rag
- Repository: https://github.com/vectorinstitute/fed-rag

## Overview

##### Project Type & Purpose
- FedRAG is a federated learning framework specifically designed for fine-tuning Retrieval-Augmented Generation (RAG) systems. The framework enables a central server and multiple client nodes to collaboratively fine-tune a common RAG model without sharing raw data. Instead, only model weight updates are exchanged.
- The README highlights that the system leverages federated fine-tuning concepts, and parts of the source (in the “src” directory) describe implementation details for generators, retrievers, and FL tasks.

##### Analyzing the File Structure
- Top-Level Directories and Files:
- Directories like “src” and “tests” represent the core implementation components and their tests.
- The “examples” subdirectories show practical example usages (quick-start and ra-dit) that can inform how end users interact with the system.
- The “.github” folder with workflows indicates continuous integration through automated linting, testing, and release procedures.
- Within “src/fed_rag”, modules are organized into several submodules such as:
- base: Contains fundamental classes for tasks, generators, retrievers, tokenizers, and trainer configurations.
- fl_tasks: Contains implementations specific to federated learning tasks (huggingface, pytorch).
- decorators and exceptions: Providing additional runtime support.
- generators, retrievers, inspectors, tokenizers: Highlighting the different processing stages in an RAG system.
- types: Includes wrappers for FL components referencing external libraries (e.g., Flower’s flwr.Client and flwr.Server).
- Config and CI files (e.g., Makefile, pre-commit, .gitignore) support development procedures rather than system design but hint at a disciplined modular structure with testing and CI/CD pipelines.

##### Key Components to Highlight
a. Federated Learning Components:
- Server Node: Representing the central aggregation point where global model updates are managed. Include components such as fl_tasks (which may include training orchestration) and “types/server.py” (server wrapper).
- Client Nodes: Represent individual nodes that run local fine-tuning on their data. Each client implements local branches of components such as data loaders (from examples), loss functions, generation, and retrieval logic. “types/client.py” should be indicated here.
b. Core RAG System Modules:
- Retrieval Component: This includes retrievers (e.g., “src/fed_rag/retrievers/hf_sentence_transformer.py”) which extract relevant context/data.
- Generation Component: Generator modules (e.g., “src/fed_rag/generators/hf_pretrained_model.py” and related base classes).
- Knowledge Store: Modules under “knowledge_stores” to manage cached or indexed information that supports the RAG process.
- Tokenizers: Essential for text preprocessing (see “src/fed_rag/tokenizers”).
- Training & Loss Modules: Include fl_task definitions in “src/fed_rag/fl_tasks”, trainer configurations, and loss functions.
c. Communication & Aggregation:
- Bidirectional flow between client nodes and the central server. This typically involves weight updates sent from clients to the server and aggregated models broadcast back.
- The use of FL wrappers (wrapping frameworks such as Flower) should be noted, ensuring that abstracted communication channels are visually represented.

##### Representing Relationships and Data Flows
- Data flow: From client components (local computation) to the aggregator in the server.
- Model update flow: “Weight updates” flowing from clients to server and “aggregated global model” flowing back.
- Internal module interactions within a node (e.g., input data feeding into tokenizers, which then interface with retrievers and generators).
- How external libraries (like Hugging Face models) integrate into the “generators”, “retrievers”, and “tokenizers” components by drawing external service/outside components or simply labeling them within a “3rd-party Libraries” section.

##### Architectural Patterns & Design Principles
- Modularity: Highlight the separation into well-defined modules (base, generators, retrievers, etc.) that can be individually updated/tested.
- Federated Learning Pattern: Capture the central server/client split, emphasizing that the server holds the common model and coordinates fine-tuning rounds.

## Diagram (Mermaid)

```mermaid
flowchart TD
    %% Federated Server Cluster
    subgraph "Federated Server"
        FS["Federated Server"]:::server
        FS_wrapper["FL Communication Wrapper"]:::server
        FS1["FL Task: HuggingFace"]:::server
        FS2["FL Task: Pytorch"]:::server
    end

    %% Client Node Cluster
    subgraph "Client Node"
        CN["Client Node"]:::client
        CN_module["Local Fine-Tuning & Data Processing"]:::client
    end

    %% RAG Components Cluster
    subgraph "RAG Components"
        TOKEN["Tokenization Module"]:::module
        RET["Retriever Module"]:::module
        GEN1["Generator Module: HF Pretrained Model"]:::module
        GEN2["Generator Module: HF PEFT Model"]:::module
        TRAIN["Trainer Configuration"]:::module
        FLTASK["FL Task Base"]:::module
        STORE["Knowledge Store"]:::module
    end

    %% Communication between Client and Server
    CN -->|"WeightUpdates"| FS
    FS -->|"GlobalModelBroadcast"| CN

    %% Internal Server Connections
    FS_wrapper --> FS1
    FS_wrapper --> FS2

    %% FL Tasks Integration
    TRAIN --> FLTASK
    FLTASK --> FS1
    FLTASK --> FS2

    %% RAG Components Flow
    CN_module -->|"LocalProcessing"| TOKEN
    TOKEN -->|"ProcessedInput"| RET
    TOKEN -->|"ProcessedInput"| GEN1
    TOKEN -->|"ProcessedInput"| GEN2
    RET -->|"SuppliesContext"| TRAIN
    GEN1 -->|"SuppliesGeneration"| TRAIN
    GEN2 -->|"SuppliesGeneration"| TRAIN
    STORE -->|"Supports"| TRAIN

    %% Click Events for Component Mapping
    click FS1 "https://github.com/vectorinstitute/fed-rag/blob/main/src/fed_rag/fl_tasks/huggingface.py"
    click FS2 "https://github.com/vectorinstitute/fed-rag/blob/main/src/fed_rag/fl_tasks/pytorch.py"
    click FS_wrapper "https://github.com/vectorinstitute/fed-rag/blob/main/src/fed_rag/types/rag_system.py"
    click RET "https://github.com/vectorinstitute/fed-rag/blob/main/src/fed_rag/retrievers/hf_sentence_transformer.py"
    click GEN1 "https://github.com/vectorinstitute/fed-rag/blob/main/src/fed_rag/generators/hf_pretrained_model.py"
    click GEN2 "https://github.com/vectorinstitute/fed-rag/blob/main/src/fed_rag/generators/hf_peft_model.py"
    click STORE "https://github.com/vectorinstitute/fed-rag/blob/main/src/fed_rag/knowledge_stores/in_memory.py"
    click TOKEN "https://github.com/vectorinstitute/fed-rag/blob/main/src/fed_rag/tokenizers/hf_pretrained_tokenizer.py"
    click TRAIN "https://github.com/vectorinstitute/fed-rag/blob/main/src/fed_rag/base/trainer_config.py"
    click FLTASK "https://github.com/vectorinstitute/fed-rag/blob/main/src/fed_rag/base/fl_task.py"

    %% Styles
    classDef server fill:#FFC0CB,stroke:#FF1493,stroke-width:2px;
    classDef client fill:#98FB98,stroke:#32CD32,stroke-width:2px;
    classDef module fill:#ADD8E6,stroke:#1E90FF,stroke-width:2px;
```
