# piy-us/rag-chatbook architecture

> Architecture diagram of the GitHub repository piy-us/rag-chatbook, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-03-31). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/piy-us/rag-chatbook
- Repository: https://github.com/piy-us/rag-chatbook

## Overview

##### Project Type and Purpose
- The project is named “Rag_Chatbook” – this hints at a system that involves retrieval-augmented generation (RAG) for chatting or interactive querying against books/documents.
- The file names (e.g., OCR_with_mouse_drag.py, pdftoimg.py, chat_engine.py, embeddings.py, vector_store.py, search.py) indicate that the project is designed to process PDFs (or images), perform OCR/cropping operations (using cropperjs assets), generate embeddings, index the extracted text, and then support a chat interface or search mechanism based on those embeddings.

##### File Structure
- Top-Level Python Files: Key modules like app.py, app2.py, chat_engine.py, config.py, data_loader.py, embeddings.py, vector_store.py, and search.py indicate the core logic and server-side functionalities.
- OCR/Conversion Scripts: Files such as OCR_with_mouse_drag.py and pdftoimg.py are likely responsible for converting PDFs to images and handling interactive image-based OCR (with cropperjs).
- Frontend Assets: Folders like “templates” and “static” (inside which there are HTML, CSS, JavaScript, and an embedded copy of the cropperjs library) suggest the presence of a web UI.
- External Assets: The cropperjs directory contains JavaScript/CSS assets for client-side image cropping which are also duplicated inside the static folder – pointing to frontend interactivity specifically for selecting cropped areas of PDFs images for OCR processing.
- Some additional PDF files (_OceanofPDF.com_The_Alchemist.pdf and extmoti.pdf) indicate sample or test documents for processing.

- Clearly label every component (e.g., “Frontend UI”, “PDF to Image Converter”, “OCR Module”, “Embeddings Generator”, “Vector Store”, “Search Service”, “Chat Engine”, “Configuration”).
- Ancillary elements such as configuration (config.py) and note sample PDF inputs (e.g., _OceanofPDF.com_The_Alchemist.pdf, extmoti.pdf) to indicate real data sources.

## Diagram (Mermaid)

```mermaid
flowchart TD
    %% Frontend Group
    subgraph "Frontend User Interface"
        WebPage("Web Pages (UI)"):::frontend
        StaticAssets("Static Assets"):::frontend
    end

    %% Backend Group
    subgraph "Backend Server"
        API("Backend API Services"):::backend
        App1("app.py"):::backend
        App2("app2.py"):::backend
        Config("Configuration"):::config
    end

    %% Document Processing Pipeline
    subgraph "Document Processing Pipeline"
        P2I("PDF to Image Converter"):::processing
        OCR("OCR Module (Interactive)"):::processing
        Cropper("CropperJS Assets"):::processing
        Loader("Data Loader"):::processing
    end

    %% Semantic Processing and Chat
    subgraph "Semantic Processing & Chat Engine"
        Embed("Embeddings Generator"):::processing
        Vector("Vector Store"):::storage
        Search("Search Service"):::processing
        Chat("Chat Engine"):::processing
    end

    %% Data Flow Connections
    WebPage -->|"HTTP Request"| API
    StaticAssets -->|"Provides UI Assets"| API

    API -->|"EntryPoint Call"| App1
    API -->|"EntryPoint Call"| App2
    API ---|"Loads Config"| Config

    API -->|"Triggers"| P2I
    P2I -->|"Converts PDF"| OCR
    OCR -->|"Utilizes"| Cropper
    OCR -->|"Processes Images"| Loader
    Loader -->|"Extracted Text"| Embed
    Embed -->|"Generates Embeddings"| Vector
    Vector -->|"Search Query"| Search
    Search -->|"Results to"| Chat
    Chat -->|"Response to UI"| WebPage

    %% Click Events
    click WebPage "https://github.com/piy-us/rag-chatbook/blob/main/templates/page_two.html"
    click StaticAssets "https://github.com/piy-us/rag-chatbook/tree/main/static/"
    click App1 "https://github.com/piy-us/rag-chatbook/blob/main/app.py"
    click App2 "https://github.com/piy-us/rag-chatbook/blob/main/app2.py"
    click P2I "https://github.com/piy-us/rag-chatbook/blob/main/pdftoimg.py"
    click OCR "https://github.com/piy-us/rag-chatbook/blob/main/OCR_with_mouse_drag.py"
    click Cropper "https://github.com/piy-us/rag-chatbook/tree/main/cropperjs/"
    click Loader "https://github.com/piy-us/rag-chatbook/blob/main/data_loader.py"
    click Embed "https://github.com/piy-us/rag-chatbook/blob/main/embeddings.py"
    click Vector "https://github.com/piy-us/rag-chatbook/blob/main/vector_store.py"
    click Search "https://github.com/piy-us/rag-chatbook/blob/main/search.py"
    click Chat "https://github.com/piy-us/rag-chatbook/blob/main/chat_engine.py"
    click Config "https://github.com/piy-us/rag-chatbook/blob/main/config.py"

    %% Styles
    classDef frontend fill:#ADD8E6,stroke:#000,stroke-width:2px;
    classDef backend fill:#FFCCCB,stroke:#000,stroke-width:2px;
    classDef processing fill:#90EE90,stroke:#000,stroke-width:2px;
    classDef storage fill:#DDA0DD,stroke:#000,stroke-width:2px;
    classDef config fill:#F5DEB3,stroke:#000,stroke-width:2px;
```
