# arjunsah21/inotebook architecture

> Architecture diagram of the GitHub repository arjunsah21/inotebook, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-05-17). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/arjunsah21/inotebook
- Repository: https://github.com/arjunsah21/inotebook

## Overview

##### Project type and purpose
- This is a Retrieval-Augmented Generation (RAG) system implemented in Python.
- It provides a Streamlit-based UI for ingesting PDF documents, indexing their contents in Milvus, and running OpenAI GPT-4 over retrieved chunks to answer user questions.

##### File structure
- Top-level scripts:
- app.py           – Streamlit application (frontend + orchestrator)
- document_processor.py – PDF/text/table/image extraction (uses PyPDF2 & Tesseract)
- vector_store.py  – Milvus client (pymilvus) for indexing & retrieval
- rag_chain.py     – LangChain RAG orchestration + conversation memory
- docker-compose.yml – Spins up Milvus standalone container
- test_milvus.py   – Connectivity/indexing tests for the vector store
- Config files: .env.example, requirements.txt

##### README details
- PDF processing, Milvus vector database, OpenAI GPT-4, Streamlit UI
- Setup: pip install, .env for API keys, Docker-Compose for Milvus, Tesseract OCR

##### Main components & interactions
a. Components
- Streamlit UI (app.py)
- PDF Processor (document_processor.py)
- Vector Store (vector_store.py → Milvus container)
- RAG Chain (rag_chain.py)
- External Services: OpenAI GPT-4 API, Tesseract OCR engine
- Configuration (.env)

b. Relationships & Data Flow
1. User uploads PDF in Streamlit UI → app.py
2. app.py calls document_processor.py → extracts text/tables/images → chunks content
3. document_processor.py calls vector_store.py → generates embeddings → indexes in Milvus
4. On user query, app.py calls rag_chain.py → vector_store.py retrieves top-k chunks → rag_chain.py calls OpenAI API (with conversation memory) → returns answer to app.py → displayed in UI

c. Architectural patterns
- Pipeline pattern: Ingestion → Indexing → Retrieval → Generation
- Separation of concerns: each Python module has a single responsibility
- External microservice: Milvus runs separately (Docker Compose)

d. Key technologies
- Streamlit (UI)
- PyPDF2 & Tesseract OCR (PDF extraction)
- pymilvus (Milvus client)
- LangChain (RAG orchestration + memory)
- OpenAI Python SDK (GPT-4 calls)
- Docker Compose (Milvus deployment)

##### Detailed component callouts
- Test_milvus.py off to the side as a verification utility feeding into vector_store.py

## Diagram (Mermaid)

```mermaid
graph TB
    %% Frontend Layer
    subgraph "Frontend"
        direction TB
        UI["app.py"]:::frontend
    end

    %% Processing Layer
    subgraph "Processing Modules"
        direction TB
        DP["document_processor.py"]:::processing
        DP -->|uses Tesseract OCR| TESS["Tesseract OCR"]:::external
    end

    %% Storage Layer
    subgraph "Storage"
        direction TB
        VS["vector_store.py"]:::storage
        VS -->|"indexes & retrieves"| MIL["Milvus\n(container)"]:::service
    end

    %% Retrieval & Generation
    subgraph "RAG"
        direction TB
        RC["rag_chain.py\n(LangChain + Memory)"]:::processing
    end

    %% External APIs
    subgraph "External Services"
        direction TB
        OA["OpenAI GPT-4 API"]:::external
    end

    %% Utilities & Config
    subgraph "Utilities & Config"
        direction TB
        TM["test_milvus.py"]:::utility
        ENV[".env.example"]:::utility
        REQ["requirements.txt"]:::utility
        RD["README.md"]:::utility
    end

    %% Data Flow
    UI -->|"upload PDF"| DP
    DP -->|"extract & chunk"| VS
    UI -->|"user query"| RC
    RC -->|"retrieve top-k vectors"| VS
    RC -->|"call GPT-4"| OA
    OA -->|"answer"| UI

    %% Test utility
    TM -->|"tests connectivity"| VS

    %% Click events
    click UI "https://github.com/arjunsah21/inotebook/blob/main/app.py"
    click DP "https://github.com/arjunsah21/inotebook/blob/main/document_processor.py"
    click VS "https://github.com/arjunsah21/inotebook/blob/main/vector_store.py"
    click RC "https://github.com/arjunsah21/inotebook/blob/main/rag_chain.py"
    click MIL "https://github.com/arjunsah21/inotebook/blob/main/docker-compose.yml"
    click TM "https://github.com/arjunsah21/inotebook/blob/main/test_milvus.py"
    click ENV "https://github.com/arjunsah21/inotebook/blob/main/.env.example"
    click REQ "https://github.com/arjunsah21/inotebook/blob/main/requirements.txt"
    click RD "https://github.com/arjunsah21/inotebook/blob/main/README.md"

    %% Styles
    classDef frontend fill:#AEDFF7,stroke:#0366d6,color:#000000
    classDef processing fill:#D4F7A8,stroke:#2E8B57,color:#000000
    classDef storage fill:#FFD59E,stroke:#FF8C00,color:#000000
    classDef service fill:#FFD59E,stroke:#FF8C00,color:#000000
    classDef external fill:#E2E2E2,stroke:#555555,color:#000000
    classDef utility fill:#F0F0F0,stroke:#888888,color:#000000
```
