How ictnlp/auto-rag works
Main Components
a. Auto-RAG Core Module:
• This is represented by the files within the Auto-RAG directory: “autorag.py,” “prompts.py,” and “gui.py.”
• It embodies the core logic for iterative retrieval interaction, decision-making, and answer generation.
b. LLM Inference Component:
• The deployed language model (using vLLM as described in the deployment scripts). This component handles model inference.
c. Retriever Component:
• External to the core module, likely based on methods from the DPR project.
• It interacts with a knowledge base (in this case, the wiki corpus) using the configuration (e.g., FAISS index and dense retriever setup).
d. GUI Layer:
• Provided by “gui.py” and supported via scripts (e.g., run_gui.sh), this component offers a dashboard or browser-based interface that allows users to input queries and view the iterative dialogue between the Auto-RAG and the retriever.
e. FlexRAG Integration:
• As shown by the instructions in the README, Auto-RAG can operate as a FlexRAG assistant. This indicates integration points where Auto-RAG acts as a pluggable module within a larger retrieval-generation framework.
f. Scripts & Deployment:
• Deployment scripts (deploy.sh, prepare_retriever.sh, run_evaluation.sh, run_gui.sh) tie all components together, ensuring proper initialization and communication among the components. They are critical for configuring the environment, model parameters, and integrations (e.g., vLLM’s GPU settings).
Relationships and Interactions
• User Interaction:
• The user accesses the GUI (or initiates a command-line session) which triggers GUI components in the Auto-RAG module.
• Core Processing:
• The Auto-RAG core module processes the input and decides if additional retrieval is needed.
• It sends queries to the Retriever component.
• Retrieval Iteration:
• The retriever (using pre-built indices and configurations defined in the scripts) returns relevant external knowledge.
• The Auto-RAG core incorporates this information in an iterative dialogue with the LLM component.
• LLM Inference:
• The language model (deployed with vLLM) processes inputs based on the iterative dialogue and generates final answers.
• Optional FlexRAG Integration:
• For users running Auto-RAG as a FlexRAG assistant, there is an integration layer where Auto-RAG’s config and responses are piped into the broader FlexRAG framework.
Architectural Patterns and Design Principles
• Modular Design:
• Each component (GUI, core logic, retriever, LLM inference) is separated into individual modules or scripts for clarity and maintainability.
• Iterative / Autonomous Loop:
• The system is designed to perform iterative queries until it is satisfied with the external knowledge that has been gathered.
• Loose Coupling:
• There is a clear separation between the client layer (GUI) and the backend (LLM, retriever), which aids in deployment and testing.
Relevant Technologies and Frameworks
• Python 3.12
• flexRAG for integration
• vLLM for model deployment/inference
• FAISS (for indexing) and DPR (for retriever setup)
• Shell scripting for deployment and evaluation processes