How intellabs/fastrag works
Project Type and Purpose
• fastRAG is an open-source research framework for building efficient retrieval-augmented generation (RAG) pipelines.
• It combines modular IR components (retrievers, rankers, document stores) with generative LLM backends (FiD, REPLUG, Gaudi, OpenVINO, Llama-CPP).
• Designed for extensibility (Haystack compatibility), optimized inference (Intel IPEX, Gaudi, ONNX), and multiple storage backends (Elasticsearch, Faiss, Qdrant, PLAID).
File-Structure Analysis
• Top-level:
• fastrag/ ← core Python package
• config/ ← YAML pipeline/component definitions
• scripts/ ← pipeline generation, indexing, training, quantization
• benchmarks/ ← evaluation scripts for BEIR, KILT
• assets/, examples/ ← sample data & notebooks
• README.md, setup.py, etc.
• Within fastrag/:
• agents/ ← conversational agent orchestrator + memory + tools
• embedders/, retrievers/, rankers/, stores/ ← IR components
• generators/ ← reader/generation backends (FiD, REPLUG, ONNX, Gaudi, Llama-CPP)
• prompt_builders/, prompt_compressors/ ← prompt-engineering modules
• ui/ ← Chainlit web UIs for demos
a. Main Components
• Configuration Layer: YAML files in config/ define pipeline graphs.
• Pipeline Builder: scripts/generate_pipeline.py + Haystack-compatible factory methods read configs.
• IR Tier:
• Document Stores (Elasticsearch, Faiss, Qdrant, PLAID)
• Embedders & Retrievers (SBERT, BM25, ColBERT-v2)
• Rankers (cross-encoders, sparse models)
• Generation Tier:
• Reader / LLM wrappers (Fusion-in-Decoder, REPLUG)
• Hardware-accelerated backends (IPEX, Optimum-Intel, ONNX, Gaudi)
• Alternative backends (OpenVINO, Llama-CPP)
• Agent & Orchestration:
• AgentStep, tools & memory (fastrag/agents)
• Prompt builders & compressors
• User Interfaces:
• Chainlit UI modules (chat, multi-modal, pipelines)
• CLI / Python API
• Auxiliary:
• Scripts for indexing, training, quantization, benchmarks
b. Relationships and Data Flows
1. User → Config Loader → Pipeline Builder
2. Pipeline Builder instantiates:
• DocumentStore ← data loading/indexing scripts
• Retriever → initial document embeddings or BM25 scores
• Ranker → refines top documents
• Generator → LLM backend produces answer
3. Agent orchestrates retrieval/generation sequence, manages conversation memory, calls tools if needed.
4. UI or Python API receives final output; benchmarks and scripts invoke pipelines programmatically.
c. Architectural Patterns
• Plugin / Factory: component classes loaded based on YAML configs
• Pipeline / DAG: sequential IR → ranking → generation stages
• Separation of Concerns: distinct modules for storage, retrieval, ranking, generation, UI
• Extensibility Points: custom embedders, retrievers, generators via clear module hierarchy
d. Key Technologies
• Haystack v2+ for pipeline abstractions
• HuggingFace Transformers / Optimum Intel / Optimum-Habana
• Intel IPEX, ONNX Runtime, OpenVINO, Gaudi accelerators, Llama-CPP
• Elasticsearch, Faiss (CPU/GPU), Qdrant, PLAID
Essential Elements to Capture
• Clear, descriptive labels for every component and storage backend.