# migue8gl/rag-agent architecture

> Architecture diagram of the GitHub repository migue8gl/rag-agent, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-05-14). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/migue8gl/rag-agent
- Repository: https://github.com/migue8gl/rag-agent

## Overview

##### Project Type and Purpose
- RAG-Agent is a full-stack, modular framework for Retrieval-Augmented Generation (RAG).
- It exposes an interactive Streamlit UI (chat_app.py) and orchestrates components—Chunker, Retriever, VectorDB—plus an LLM back end (OpenAI).

##### Main Components
a. Frontend
- Streamlit App (chat_app.py): accepts user queries, displays responses.
b. Orchestrator / Agent
- LangGraph‐based agent defined in agent/nodes/chatbot.py
- Agent tools (agent/tools/rag.py): RAG “tool” node that invokes components.
- Prompts module (agent/prompts/prompts.py) & schemas (agent/schemas).
c. Core Components
- Chunker (components/chunker.py)
- Retriever (components/retriever.py)
- VectorDB (components/vectordb.py)
d. Extension Layers (Interface → Strategy)
- Interfaces (interfaces/*_interface.py) define generic APIs.
- Strategies (strategies/*_strategies.py) implement these interfaces.
e. Configuration & Utilities
- core/config.py for settings (e.g., model & database paths).
- utils/utils.py for shared helpers.
f. Data Stores
- Raw documents (data/docs/**/*.md)
- Precomputed indices (data/bm25.pkl)
- Semantic database (data/semantic_db.pkl)
g. External Services
- OpenAI API (LLM)
- FAISS (embedded within VectorDB)
- Sentence-Transformers

##### Relationships & Data Flow
1. Raw Docs → Chunker → embeddings → VectorDB (persist).
2. BM25 index built or loaded from data/bm25.pkl.
- Query / Runtime Flow (core focus)
1. User → Streamlit UI → Orchestrator
2. Orchestrator → Retriever (via interface & strategy)
3. Retriever → VectorDB (similarity search) → Top-K chunks
4. Orchestrator → RAG Tool (patches chunks + prompt)
5. RAG Tool → OpenAI API → generated answer
6. Orchestrator returns answer → Streamlit UI → User

##### Architectural Patterns & Design Principles
- Strategy Pattern: Interfaces ↔ Strategies decouple contract from implementation.
- Modular Design: “components” folder holds core logic; “agent” folder orchestrates high-level flow.
- Separation of Concerns: UI (chat_app.py) vs. orchestration (agent) vs. core services vs. data persistence.

##### Final Checklist
- [ ] Frontend (Streamlit)
- [ ] Orchestrator & Agent Nodes
- [ ] Core Components (Chunker, Retriever, VectorDB)
- [ ] Interfaces & Strategies grouping
- [ ] Data Stores (raw docs, indices, embeddings)
- [ ] External APIs (OpenAI, FAISS backend)

## Diagram (Mermaid)

```mermaid
flowchart TD
    %% Frontend Layer
    subgraph "Frontend Layer"
        UI["Streamlit App<br/>(chat_app.py)"]:::service
    end

    %% Orchestrator Layer
    subgraph "Orchestrator Layer"
        Agent["Agent<br/>(agent/nodes/chatbot.py)"]:::service
        Tool["RAG Tool<br/>(agent/tools/rag.py)"]:::service
        Prompts["Prompts<br/>(agent/prompts/prompts.py)"]:::service
        Schemas["Schemas<br/>(agent/schemas)"]:::service
    end

    %% Core Components
    subgraph "Core Components"
        Chunker["Chunker<br/>(components/chunker.py)"]:::service
        Retriever["Retriever<br/>(components/retriever.py)"]:::service
        VectorDB["VectorDB<br/>(components/vectordb.py)"]:::service
    end

    %% Customization Layer
    subgraph "Customization"
        subgraph Interfaces
            SChunker["Chunker Interface<br/>(interfaces/chunker_interface.py)"]:::service
            SRetriever["Retriever Interface<br/>(interfaces/retriever_interface.py)"]:::service
            SVectorDB["VectorDB Interface<br/>(interfaces/vectordb_interface.py)"]:::service
        end
        subgraph Strategies
            StChunker["Chunker Strategies<br/>(strategies/chunker_strategies.py)"]:::service
            StRetriever["Retriever Strategies<br/>(strategies/retriever_strategies.py)"]:::service
        end
    end

    %% Config & Utils
    subgraph "Config & Utils"
        Config["Config<br/>(core/config.py)"]:::service
        Utils["Utils<br/>(utils/utils.py)"]:::service
    end

    %% Data Stores
    subgraph "Data Stores"
        RawDocs["Raw Docs<br/>(data/docs/)"]:::datastore
        BM25["BM25 Index<br/>(data/bm25.pkl)"]:::datastore
        SemanticDB["Semantic DB<br/>(data/semantic_db.pkl)"]:::datastore
    end

    %% External Services
    subgraph "External Services"
        OpenAI["OpenAI API"]:::external
        FAISS["FAISS"]:::external
        ST["Sentence-Transformers"]:::external
    end

    %% Ingestion Flow
    RawDocs -->|"chunk()"| Chunker
    Chunker -->|"embed(chunks)"| VectorDB
    VectorDB -->|"persist"| SemanticDB
    raw2["Load BM25"] --> BM25

    %% Query Flow
    UI -->|"user query"| Agent
    Agent -->|"search(query)"| Retriever
    Retriever -->|"similarity_search"| VectorDB
    VectorDB -->|"top-K chunks"| Agent
    Agent -->|"invoke RAG"| Tool
    Tool -->|"call OpenAI"| OpenAI
    Agent -->|"return answer"| UI

    %% Strategy Pattern Links
    Retriever -->|implements| SRetriever
    SRetriever -->|strategy| StRetriever
    Chunker -->|implements| SChunker
    SChunker -->|strategy| StChunker
    VectorDB -->|implements| SVectorDB

    %% Agent internals
    Agent --> Prompts
    Agent --> Schemas
    Agent --> Tool

    %% Config & Utils usage
    Agent --> Config
    Chunker --> Config
    Retriever --> Config
    VectorDB --> Config
    Agent --> Utils
    Chunker --> Utils
    Retriever --> Utils
    VectorDB --> Utils

    %% External integrations
    VectorDB -->|uses| FAISS
    Chunker -->|uses| ST

    %% Click Events
    click UI "https://github.com/migue8gl/rag-agent/blob/master/chat_app.py"
    click Agent "https://github.com/migue8gl/rag-agent/blob/master/agent/nodes/chatbot.py"
    click Tool "https://github.com/migue8gl/rag-agent/blob/master/agent/tools/rag.py"
    click Prompts "https://github.com/migue8gl/rag-agent/blob/master/agent/prompts/prompts.py"
    click Schemas "https://github.com/migue8gl/rag-agent/blob/master/agent/schemas/states.py"
    click Chunker "https://github.com/migue8gl/rag-agent/blob/master/components/chunker.py"
    click Retriever "https://github.com/migue8gl/rag-agent/blob/master/components/retriever.py"
    click VectorDB "https://github.com/migue8gl/rag-agent/blob/master/components/vectordb.py"
    click SChunker "https://github.com/migue8gl/rag-agent/blob/master/interfaces/chunker_interface.py"
    click SRetriever "https://github.com/migue8gl/rag-agent/blob/master/interfaces/retriever_interface.py"
    click SVectorDB "https://github.com/migue8gl/rag-agent/blob/master/interfaces/vectordb_interface.py"
    click StChunker "https://github.com/migue8gl/rag-agent/blob/master/strategies/chunker_strategies.py"
    click StRetriever "https://github.com/migue8gl/rag-agent/blob/master/strategies/retriever_strategies.py"
    click Config "https://github.com/migue8gl/rag-agent/blob/master/core/config.py"
    click Utils "https://github.com/migue8gl/rag-agent/blob/master/utils/utils.py"
    click RawDocs "https://github.com/migue8gl/rag-agent/tree/master/data/docs/"
    click BM25 "https://github.com/migue8gl/rag-agent/blob/master/data/bm25.pkl"
    click SemanticDB "https://github.com/migue8gl/rag-agent/blob/master/data/semantic_db.pkl"

    %% Styles
    classDef service fill:#AED6F1,stroke:#1F618D,color:#1B2631;
    classDef datastore fill:#ABEBC6,stroke:#196F3D,color:#1B2631;
    classDef external fill:#F5B041,stroke:#B9770E,color:#1B2631;
```
