How vectorinstitute/fed-rag works
Project Type & Purpose
• FedRAG is a federated learning framework specifically designed for fine-tuning Retrieval-Augmented Generation (RAG) systems. The framework enables a central server and multiple client nodes to collaboratively fine-tune a common RAG model without sharing raw data. Instead, only model weight updates are exchanged.
• The README highlights that the system leverages federated fine-tuning concepts, and parts of the source (in the “src” directory) describe implementation details for generators, retrievers, and FL tasks.
Analyzing the File Structure
• Top-Level Directories and Files:
• Directories like “src” and “tests” represent the core implementation components and their tests.
• The “examples” subdirectories show practical example usages (quick-start and ra-dit) that can inform how end users interact with the system.
• The “.github” folder with workflows indicates continuous integration through automated linting, testing, and release procedures.
• Within “src/fed_rag”, modules are organized into several submodules such as:
• base: Contains fundamental classes for tasks, generators, retrievers, tokenizers, and trainer configurations.
• fl_tasks: Contains implementations specific to federated learning tasks (huggingface, pytorch).
• decorators and exceptions: Providing additional runtime support.
• generators, retrievers, inspectors, tokenizers: Highlighting the different processing stages in an RAG system.
• types: Includes wrappers for FL components referencing external libraries (e.g., Flower’s flwr.Client and flwr.Server).
• Config and CI files (e.g., Makefile, pre-commit, .gitignore) support development procedures rather than system design but hint at a disciplined modular structure with testing and CI/CD pipelines.
Key Components to Highlight
a. Federated Learning Components:
• Server Node: Representing the central aggregation point where global model updates are managed. Include components such as fl_tasks (which may include training orchestration) and “types/server.py” (server wrapper).
• Client Nodes: Represent individual nodes that run local fine-tuning on their data. Each client implements local branches of components such as data loaders (from examples), loss functions, generation, and retrieval logic. “types/client.py” should be indicated here.
b. Core RAG System Modules:
• Retrieval Component: This includes retrievers (e.g., “src/fed_rag/retrievers/hf_sentence_transformer.py”) which extract relevant context/data.
• Generation Component: Generator modules (e.g., “src/fed_rag/generators/hf_pretrained_model.py” and related base classes).
• Knowledge Store: Modules under “knowledge_stores” to manage cached or indexed information that supports the RAG process.
• Tokenizers: Essential for text preprocessing (see “src/fed_rag/tokenizers”).
• Training & Loss Modules: Include fl_task definitions in “src/fed_rag/fl_tasks”, trainer configurations, and loss functions.
c. Communication & Aggregation:
• Bidirectional flow between client nodes and the central server. This typically involves weight updates sent from clients to the server and aggregated models broadcast back.
• The use of FL wrappers (wrapping frameworks such as Flower) should be noted, ensuring that abstracted communication channels are visually represented.
Representing Relationships and Data Flows
• Data flow: From client components (local computation) to the aggregator in the server.
• Model update flow: “Weight updates” flowing from clients to server and “aggregated global model” flowing back.
• Internal module interactions within a node (e.g., input data feeding into tokenizers, which then interface with retrievers and generators).
• How external libraries (like Hugging Face models) integrate into the “generators”, “retrievers”, and “tokenizers” components by drawing external service/outside components or simply labeling them within a “3rd-party Libraries” section.
Architectural Patterns & Design Principles
• Modularity: Highlight the separation into well-defined modules (base, generators, retrievers, etc.) that can be individually updated/tested.
• Federated Learning Pattern: Capture the central server/client split, emphasizing that the server holds the common model and coordinates fine-tuning rounds.