# bangoc123/drop-rag architecture

> Architecture diagram of the GitHub repository bangoc123/drop-rag, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-04-05). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/bangoc123/drop-rag
- Repository: https://github.com/bangoc123/drop-rag

## Overview

##### Key Architectural Layers/Components
- User Interface (Frontend):
- Represented mainly by the Streamlit pages found in the “pages” directory (e.g., Interactive_Chatbot.py, Graph_Rag.py) and the main entry point in app.py.
- This layer handles user interactions such as file uploads, LLM selections, and displaying chatbot responses or graph visualizations.

- Data Processing/Pipeline Logic:
- The core logic for managing uploaded files and processing content is spread over multiple modules:
- “chunking” directory: Contains various strategies for dividing content into chunks (e.g., recursive token, semantic, fixed token, and agentic chunkers).
- “collection_management.py”, “prompt.py”, and “utils.py” provide supporting functions for organizing and processing the data.

- LLM Integration:
- The “llms” directory abstracts interactions with different Large Language Models.
- Two main components:
- “llms/onlinellms.py” for cloud-based models (Gemini and OpenAI).
- “llms/localLllms.py” for local inference via OLLAMA.

- Data Storage and Retrieval:
- The README explains that a Chroma vector store is used to store vector embeddings.
- Although a dedicated storage module isn’t obvious from a single file, functions in “graph_rag.py”, “graph_search.py”, and configuration files like “neo4j.conf” indicate:
- Vector search interactions.
- Graph-based retrieval and visualization (Graph RAG).

- External Services:
- Cloud-based LLM providers (Gemini, OpenAI) and local models for inference.
- Optionally, external vector store components (Chroma) and possibly a Neo4j graph database (given the neo4j.conf) should be shown as part of the overall ecosystem.

##### Representing Relationships and Data Flow
- The Application Core calls the “Chunking Module” (various strategies from the “chunking” folder) to preprocess documents.
- After chunking, the processed data flows to the “Storage & Retrieval” component where embeddings are generated and stored in the Chroma vector store.
- For generating responses, the application routes the query to the “LLM Integration” layer where either an online or local LLM is invoked.

##### Architectural Patterns and Key Design Principles
- Separation of Concerns: Clearly delineate responsibilities between the UI (pages and app.py), business logic (chunking, file processing utilities), and external service integrations (LLM modules and data retrieval/graph visualization modules).

##### Additional Detailed Components
- The “components” directory (e.g., notify.py) that seems to contain UI or notification logic.
- Configuration Files: Represent the neo4j.conf as a configuration block for the Graph RAG feature.
- Auxiliary Modules: Ensure utilities, prompts, and constant definitions are referenced in the architecture as operational support.

Final Implementation Approach:
- Top: User Interface (Streamlit)
- Middle: Application Core (app.py, pages, and various processing utilities)
- Bottom: Integration Layers (Chunking, LLMs, Storage & Graph Visualization)

## Diagram (Mermaid)

```mermaid
graph TD
    subgraph "User Interface (Frontend)"
        UI1["app.py (Entry)"]:::ui
        UI2["Interactive Chatbot (Streamlit)"]:::ui
        UI3["Graph RAG (Streamlit)"]:::ui
    end

    subgraph "Business Logic / Data Processing"
        BL1["Chunking Module (Various Strategies)"]:::logic
        BL2["Collection Management"]:::logic
        BL3["Prompt Generation"]:::logic
        BL4["Utilities"]:::logic
    end

    subgraph "LLM Integration"
        LLM1["Online LLM (Gemini/OpenAI)"]:::llm
        LLM2["Local LLM (OLLAMA)"]:::llm
    end

    subgraph "Data Storage & Graph Retrieval"
        DS1["Chroma Vector Store"]:::storage
        DS4["Graph Search Module"]:::storage
        DS2["Graph RAG Module"]:::storage
        DS3["Graph DB Config"]:::storage
    end

    subgraph "External Services"
        EX1["Notification Service"]:::external
    end

    %% Data flow arrows
    UI1 -->|"initiates"| BL1
    UI1 -->|"initiates"| BL2
    UI1 -->|"initiates"| BL3
    UI1 -->|"initiates"| BL4

    BL1 -->|"embeds"| DS1
    BL2 -->|"embeds"| DS1
    BL3 -->|"embeds"| DS1
    BL4 -->|"embeds"| DS1

    DS1 -->|"retrieves"| LLM1
    DS1 -->|"retrieves"| LLM2

    DS1 -->|"searches"| DS4
    DS4 -->|"aggregates"| DS2
    DS2 -->|"configured_by"| DS3

    LLM1 -->|"returns"| UI2
    LLM2 -->|"returns"| UI2

    DS2 -->|"visualizes"| UI3

    UI1 -->|"notifies"| EX1

    %% Click Events
    click UI1 "https://github.com/bangoc123/drop-rag/blob/master/app.py"
    click UI2 "https://github.com/bangoc123/drop-rag/blob/master/pages/Interactive_Chatbot.py"
    click UI3 "https://github.com/bangoc123/drop-rag/blob/master/pages/Graph_Rag.py"
    click BL1 "https://github.com/bangoc123/drop-rag/tree/master/chunking"
    click BL2 "https://github.com/bangoc123/drop-rag/blob/master/collection_management.py"
    click BL3 "https://github.com/bangoc123/drop-rag/blob/master/prompt.py"
    click BL4 "https://github.com/bangoc123/drop-rag/blob/master/utils.py"
    click LLM1 "https://github.com/bangoc123/drop-rag/blob/master/llms/onlinellms.py"
    click LLM2 "https://github.com/bangoc123/drop-rag/blob/master/llms/localLllms.py"
    click DS2 "https://github.com/bangoc123/drop-rag/blob/master/graph_rag.py"
    click DS4 "https://github.com/bangoc123/drop-rag/blob/master/graph_search.py"
    click DS3 "https://github.com/bangoc123/drop-rag/blob/master/neo4j.conf"
    click EX1 "https://github.com/bangoc123/drop-rag/blob/master/components/notify.py"

    %% Styles
    classDef ui fill:#cce5ff,stroke:#1a75ff,stroke-width:2px;
    classDef logic fill:#d4edda,stroke:#28a745,stroke-width:2px;
    classDef llm fill:#fff3cd,stroke:#ffcc00,stroke-width:2px;
    classDef storage fill:#f8d7da,stroke:#dc3545,stroke-width:2px;
    classDef external fill:#e2e3e5,stroke:#6c757d,stroke-width:2px;
```
