# akariasai/self-rag architecture

> Architecture diagram of the GitHub repository akariasai/self-rag, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-04-17). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/akariasai/self-rag
- Repository: https://github.com/akariasai/self-rag

## Overview

##### Project type and purpose
- This is an open‑source ML/AI research codebase (“Self‑RAG”) for training and running a Retrieval‑Augmented Generation framework with self‑critique. It comprises:
- Data‑creation pipelines (GPT‑4 based critic data, generator data, process_data)
- Two model training flows (Critic and Generator)
- A retrieval module (Contriever)
- An inference pipeline built on vllm for adaptive retrieval, generation, and reflection

##### Top‑level file structure and architectural clues
- data_creation/
– critic/gpt4_reward: scripts to call GPT‑4 and generate reflection tokens
– generator: scripts that assemble training data via the Critic and Retriever
– process_data: preprocessing of various QA datasets
– train_special_tokens.py: fine‑tunes an LLM to predict new “reflection” tokens
- retrieval_lm/
– passage_retrieval.py, generate_passage_embeddings.py, finetune.py, run_*.py: core retrieval + generator training + inference scripts
– src/: helper modules (beir_utils, contriever, evaluation, dist_utils, etc.)
– configuration (stage3_no_offloading_accelerate.conf) for DeepSpeed/FSDP
- environment.yml, requirements.txt: Python stack (PyTorch, Hugging Face Transformers, vllm, DeepSpeed, Contriever)
- setup.sh: helper to bootstrap environment

##### Main system components
A. Data Creation Layer
1. External Service: GPT‑4 API
2. Critic Data Generator (data_creation/critic)
3. Generator Data Generator (data_creation/generator)
4. Dataset Preprocessor (data_creation/process_data)
B. Training Layer
1. Critic Training (train_special_tokens.py → outputs Critic model)
2. Generator Training (retrieval_lm/finetune.py + DeepSpeed scripts → outputs Generator model)
C. Retrieval Service
1. Passage Corpus + Embedding DB (download_demo_corpus.sh + generated embeddings)
2. Retriever Module (Contriever implementation in retrieval_lm/src/contriever.py + passage_retrieval.py)
D. Inference Layer
1. Orchestrator (run_short_form.py, run_long_form_static.py)
2. vllm Engine (LLM wrapper, sampling with reflection tokens)
3. Adaptive Retrieval Loop (threshold logic, multiple retrieval rounds)
4. Segment‑level Beam Search (utility, relevance, support weights)

##### Relationships and interactions
- Data Creation Layer → feeds training data to Critic Training
- Critic model + Retrieval Service → feed into Generator Data Creation
- Generator model → training output, stored for inference
- Inference Orchestrator loads Generator model, calls Retriever service on demand, loops between retrieval and generation with Critic tokens embedded in generation
- All scripts share common utility modules in retrieval_lm/src/ and rely on environment.yml / requirements.txt to maintain reproducibility

##### Architectural patterns and design principles
- Layered pipeline: clear separation of data‑creation, training, retrieval, and inference
- Modular microservices: GPT‑4 calls, Contriever retriever, vllm inference, DeepSpeed training
- Extensibility: ability to swap retrieval model, LLM checkpoints, or fine‑tuning scripts by changing parameters or conda/Python dependencies
- Reproducibility & scalability: environment.yml, DeepSpeed/FSDP configs, script_finetune_*.sh

##### Key technologies, frameworks, and libraries
- Python 3.x, PyTorch, Hugging Face Transformers, vllm
- DeepSpeed (FSDP, config files), Torchrun
- Contriever for retrieval embeddings & search
- GPT‑4/OpenAI API for data labelling
- Conda / pip environment management

## Diagram (Mermaid)

```mermaid
flowchart TB
    %% External Service
    GPT4["GPT-4 API"]:::external

    subgraph "Data Creation Layer"
        CDG["Critic Data Generator"]:::data
        GDG["Generator Data Generator"]:::data
        DP["Dataset Preprocessor"]:::data
    end

    subgraph "Training Layer"
        CT["Critic Training\n(train_special_tokens.py)"]:::train
        GT["Generator Training\n(finetune.py)"]:::train
    end

    subgraph "Retrieval Service"
        PC["Passage Corpus & Embedding DB"]:::retrieval
        RM["Retriever Module"]:::retrieval
    end

    subgraph "Inference Layer"
        ORC["Orchestrator\n(run_short_form.py,\nrun_long_form_static.py)"]:::inference
        VLLM["vllm Engine"]:::inference
        AR["Adaptive Retrieval Loop"]:::inference
    end

    %% Data Creation flows
    GPT4 -->|provides data| CDG
    CDG -->|critic data| CT
    DP -->|processed datasets| GDG
    CDG -->|critic model inputs| GDG

    %% Training flows
    CT -->|outputs Critic model| GDG
    GDG -->|generator data| GT

    %% Retrieval Service interactions
    PC --> RM
    RM -->|retrieval for data_gen| GDG
    RM -->|on-demand retrieval| ORC

    %% Inference flows
    GT -->|Generator model| ORC
    ORC -->|requests generation| VLLM
    VLLM -->|reflect tokens| AR
    AR -->|retrieval loop| RM

    %% Click Events
    click CDG "https://github.com/akariasai/self-rag/tree/main/data_creation/critic/gpt4_reward/"
    click GDG "https://github.com/akariasai/self-rag/tree/main/data_creation/generator/"
    click DP "https://github.com/akariasai/self-rag/tree/main/data_creation/process_data/"
    click CT "https://github.com/akariasai/self-rag/blob/main/data_creation/train_special_tokens.py"
    click GT "https://github.com/akariasai/self-rag/blob/main/retrieval_lm/finetune.py"
    click PC "https://github.com/akariasai/self-rag/blob/main/retrieval_lm/download_demo_corpus.sh"
    click RM "https://github.com/akariasai/self-rag/blob/main/retrieval_lm/src/contriever.py"
    click ORC "https://github.com/akariasai/self-rag/blob/main/retrieval_lm/run_short_form.py"
    click VLLM "https://github.com/akariasai/self-rag/blob/main/retrieval_lm/run_baseline_lm.py"
    click AR "https://github.com/akariasai/self-rag/blob/main/retrieval_lm/src/utils.py"

    %% Styles
    classDef data fill:#ADD8E6,stroke:#000,stroke-width:1px;
    classDef train fill:#90EE90,stroke:#000,stroke-width:1px;
    classDef retrieval fill:#FFA500,stroke:#000,stroke-width:1px;
    classDef inference fill:#DDA0DD,stroke:#000,stroke-width:1px;
    classDef external fill:#CCCCCC,stroke:#000,stroke-width:1px;
```
