How migue8gl/rag-agent works
Project Type and Purpose
• RAG-Agent is a full-stack, modular framework for Retrieval-Augmented Generation (RAG).
• It exposes an interactive Streamlit UI (chat_app.py) and orchestrates components—Chunker, Retriever, VectorDB—plus an LLM back end (OpenAI).
Main Components
a. Frontend
• Streamlit App (chat_app.py): accepts user queries, displays responses.
b. Orchestrator / Agent
• LangGraph‐based agent defined in agent/nodes/chatbot.py
• Agent tools (agent/tools/rag.py): RAG “tool” node that invokes components.
• Prompts module (agent/prompts/prompts.py) & schemas (agent/schemas).
c. Core Components
• Chunker (components/chunker.py)
• Retriever (components/retriever.py)
• VectorDB (components/vectordb.py)
d. Extension Layers (Interface → Strategy)
• Interfaces (interfaces/*_interface.py) define generic APIs.
• Strategies (strategies/*_strategies.py) implement these interfaces.
e. Configuration & Utilities
• core/config.py for settings (e.g., model & database paths).
• utils/utils.py for shared helpers.
f. Data Stores
• Raw documents (data/docs/**/*.md)
• Precomputed indices (data/bm25.pkl)
• Semantic database (data/semantic_db.pkl)
g. External Services
• OpenAI API (LLM)
• FAISS (embedded within VectorDB)
• Sentence-Transformers
Relationships & Data Flow
1. Raw Docs → Chunker → embeddings → VectorDB (persist).
2. BM25 index built or loaded from data/bm25.pkl.
• Query / Runtime Flow (core focus)
1. User → Streamlit UI → Orchestrator
2. Orchestrator → Retriever (via interface & strategy)
3. Retriever → VectorDB (similarity search) → Top-K chunks
4. Orchestrator → RAG Tool (patches chunks + prompt)
5. RAG Tool → OpenAI API → generated answer
6. Orchestrator returns answer → Streamlit UI → User
Architectural Patterns & Design Principles
• Strategy Pattern: Interfaces ↔ Strategies decouple contract from implementation.
• Modular Design: “components” folder holds core logic; “agent” folder orchestrates high-level flow.
• Separation of Concerns: UI (chat_app.py) vs. orchestration (agent) vs. core services vs. data persistence.
Final Checklist
• [ ] Frontend (Streamlit)
• [ ] Orchestrator & Agent Nodes
• [ ] Core Components (Chunker, Retriever, VectorDB)
• [ ] Interfaces & Strategies grouping
• [ ] Data Stores (raw docs, indices, embeddings)
• [ ] External APIs (OpenAI, FAISS backend)