# raisatahasen/rag architecture

> Architecture diagram of the GitHub repository raisatahasen/rag, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-08-04). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/raisatahasen/rag
- Repository: https://github.com/raisatahasen/rag

## Overview

##### Project type and purpose
- This is a full-stack, bilingual Retrieval-Augmented Generation (RAG) pipeline.
- It ingests a Bangla PDF document, chunks and embeds its content, stores vectors in a vector database, retrieves relevant chunks on user queries (Bangla or English), and then generates context-aware answers via an LLM exposed over a simple web API with a chat UI.

##### File structure
Top level:
- .env, requirements.txt – environment variables and dependencies
- README.md, Setup Guide.pdf, RAG_Q_A.pdf – docs and assessment materials
- api.py, chat_ui.html, main.py – API and UI entry points
- preprocessor.py, embedder.py, vector_store.py, retriever.py, generator.py – core pipeline stages
- data/ – contains source PDF (HSC26-Bangla1st-Paper.pdf)
- RAG_Q_A.pdf, Screenshot_of_DemoChat – reference/test files

##### Additional insights from README
- Uses Flask + Flask-CORS for the web API.
- Data pipeline leverages PyMuPDF, LlamaIndex for chunking, SentenceTransformers for embeddings (paraphrase-multilingual-mpnet-base-v2), ChromaDB (via LangChain) for vector storage, and OpenRouter to call an LLM (Anthropic/Claude-3).
- Session-aware API: clients POST {query, session_id} to /chat; context stored in memory.

##### Detail
- Environment config (.env) feeding API & services with API keys.
- Optional annotation of background scripts (e.g., a one-time `main.py` or `vector_store.py` job to build the index).
- Interfaces (HTTP REST) and data formats (JSON for /chat).

## Diagram (Mermaid)

```mermaid
flowchart TD
    %% Frontend & API Layer
    subgraph "Frontend & API Layer"
        direction TB
        UI["Chat UI"]:::apiui
        API["Flask API (/chat)"]:::apiui
        Env[".env"]:::apiui
        API -->|"loads config"| Env
        UI -->|"AJAX POST {query, session_id}"| API
        API -->|"maintains session store (in-memory)"| API
    end

    %% Data Processing Pipeline (Offline)
    subgraph "Offline Indexing Pipeline"
        direction TB
        PDF["HSC26-Bangla1st-Paper.pdf"]:::storage
        Main["main.py"]:::pipeline
        Preprocessor["PDF Chunker\n(preprocessor.py)"]:::pipeline
        Embedder["Embedding Service\n(embedder.py)"]:::pipeline
        VectorStore["Vector Store\n(vector_store.py)"]:::storage
        PDF --> Main
        Main --> Preprocessor
        Preprocessor --> Embedder
        Embedder --> VectorStore
    end

    %% Vector DB & Retrieval
    subgraph "Storage & Retrieval"
        direction TB
        Retriever["Retrieval Service\n(retriever.py)"]:::storage
        DB[(ChromaDB)]:::storage
        VectorStore --> DB
        API -->|"calls retrieval"| Retriever
        Retriever -->|"semantic search"| DB
        DB --> Retriever
    end

    %% Generation Service
    subgraph "Generation Service"
        direction TB
        Generator["LLM Generator\n(generator.py)"]:::generation
        ExternalLLM["OpenRouter\n(LLM Endpoint)"]:::external
        API -->|"calls generator"| Generator
        Generator -->|"API call"| ExternalLLM
        ExternalLLM --> Generator
    end

    %% Online Flow Response
    Generator -->|"response {answer}"| API
    API -->|"JSON {answer}"| UI

    %% Documentation & Dependencies
    subgraph "Docs & Config"
        direction TB
        Req["requirements.txt"]:::docs
        Readme["README.md"]:::docs
        Setup["Setup Guide.pdf"]:::docs
        RagQA["RAG_Q_A.pdf"]:::docs
    end

    %% Click Events
    click Preprocessor "https://github.com/raisatahasen/rag/blob/main/preprocessor.py"
    click Embedder "https://github.com/raisatahasen/rag/blob/main/embedder.py"
    click VectorStore "https://github.com/raisatahasen/rag/blob/main/vector_store.py"
    click Retriever "https://github.com/raisatahasen/rag/blob/main/retriever.py"
    click Generator "https://github.com/raisatahasen/rag/blob/main/generator.py"
    click API "https://github.com/raisatahasen/rag/blob/main/api.py"
    click UI "https://github.com/raisatahasen/rag/blob/main/chat_ui.html"
    click Main "https://github.com/raisatahasen/rag/blob/main/main.py"
    click PDF "https://github.com/raisatahasen/rag/blob/main/data/HSC26-Bangla1st-Paper.pdf"
    click Env "https://github.com/raisatahasen/rag/blob/main/.env"
    click Req "https://github.com/raisatahasen/rag/blob/main/requirements.txt"
    click Readme "https://github.com/raisatahasen/rag/blob/main/README.md"
    click Setup "https://github.com/raisatahasen/rag/blob/main/Setup Guide.pdf"
    click RagQA "https://github.com/raisatahasen/rag/blob/main/RAG_Q_A.pdf"

    %% Styles
    classDef pipeline fill:#ADD8E6,stroke:#000,stroke-width:1px;
    classDef storage fill:#90EE90,stroke:#000,stroke-width:1px;
    classDef generation fill:#FFA500,stroke:#000,stroke-width:1px;
    classDef apiui fill:#D8BFD8,stroke:#000,stroke-width:1px;
    classDef external fill:#FF6347,stroke:#000,stroke-width:1px;
    classDef docs fill:#F0E68C,stroke:#000,stroke-width:1px;
```
