How akariasai/self-rag works
Project type and purpose
• This is an open‑source ML/AI research codebase (“Self‑RAG”) for training and running a Retrieval‑Augmented Generation framework with self‑critique. It comprises:
• Data‑creation pipelines (GPT‑4 based critic data, generator data, process_data)
• Two model training flows (Critic and Generator)
• A retrieval module (Contriever)
• An inference pipeline built on vllm for adaptive retrieval, generation, and reflection
Top‑level file structure and architectural clues
• data_creation/
– critic/gpt4_reward: scripts to call GPT‑4 and generate reflection tokens
– generator: scripts that assemble training data via the Critic and Retriever
– process_data: preprocessing of various QA datasets
– train_special_tokens.py: fine‑tunes an LLM to predict new “reflection” tokens
• retrieval_lm/
– passage_retrieval.py, generate_passage_embeddings.py, finetune.py, run_*.py: core retrieval + generator training + inference scripts
– src/: helper modules (beir_utils, contriever, evaluation, dist_utils, etc.)
– configuration (stage3_no_offloading_accelerate.conf) for DeepSpeed/FSDP
• environment.yml, requirements.txt: Python stack (PyTorch, Hugging Face Transformers, vllm, DeepSpeed, Contriever)
• setup.sh: helper to bootstrap environment
Main system components
A. Data Creation Layer
1. External Service: GPT‑4 API
2. Critic Data Generator (data_creation/critic)
3. Generator Data Generator (data_creation/generator)
4. Dataset Preprocessor (data_creation/process_data)
B. Training Layer
1. Critic Training (train_special_tokens.py → outputs Critic model)
2. Generator Training (retrieval_lm/finetune.py + DeepSpeed scripts → outputs Generator model)
C. Retrieval Service
1. Passage Corpus + Embedding DB (download_demo_corpus.sh + generated embeddings)
2. Retriever Module (Contriever implementation in retrieval_lm/src/contriever.py + passage_retrieval.py)
D. Inference Layer
1. Orchestrator (run_short_form.py, run_long_form_static.py)
2. vllm Engine (LLM wrapper, sampling with reflection tokens)
3. Adaptive Retrieval Loop (threshold logic, multiple retrieval rounds)
4. Segment‑level Beam Search (utility, relevance, support weights)
Relationships and interactions
• Data Creation Layer → feeds training data to Critic Training
• Critic model + Retrieval Service → feed into Generator Data Creation
• Generator model → training output, stored for inference
• Inference Orchestrator loads Generator model, calls Retriever service on demand, loops between retrieval and generation with Critic tokens embedded in generation
• All scripts share common utility modules in retrieval_lm/src/ and rely on environment.yml / requirements.txt to maintain reproducibility
Architectural patterns and design principles
• Layered pipeline: clear separation of data‑creation, training, retrieval, and inference
• Modular microservices: GPT‑4 calls, Contriever retriever, vllm inference, DeepSpeed training
• Extensibility: ability to swap retrieval model, LLM checkpoints, or fine‑tuning scripts by changing parameters or conda/Python dependencies
• Reproducibility & scalability: environment.yml, DeepSpeed/FSDP configs, script_finetune_*.sh
Key technologies, frameworks, and libraries
• Python 3.x, PyTorch, Hugging Face Transformers, vllm
• DeepSpeed (FSDP, config files), Torchrun
• Contriever for retrieval embeddings & search
• GPT‑4/OpenAI API for data labelling
• Conda / pip environment management