# mildsauce9/documentor architecture

> Architecture diagram of the GitHub repository mildsauce9/documentor, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-10-21). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/mildsauce9/documentor
- Repository: https://github.com/mildsauce9/documentor

## Overview

##### Project type and purpose
- DocuMentor is a full-stack web application that provides an intelligent, Retrieval-Augmented Generation (RAG) pipeline for natural-language Q&A over uploaded PDF documents.
- It consists of a frontend UI (static HTML/JS/CSS), a Python-based backend (app.py), local session persistence (file system), and calls out to the OpenAI API for embeddings and completions.

##### File structure
- Top-level files
- Dockerfile, .dockerignore – containerization
- requirements.txt – Python dependencies (likely includes streamlit or Flask, faiss-cpu, OpenAI SDK, PDF parser)
- app.py – main backend entrypoint
- README.md – project overview and setup
- static/
- index.html, app.js, styles.css – frontend assets served to the browser
- user_sessions/
- {session_uuid}/chat_history.json – persisted conversation turns
- {session_uuid}/faiss_index/index.faiss & index.pkl – Faiss vector store on disk

##### README for additional insights
- Describes RAG features: PDF upload, contextual memory, source-aware answers
- Instructs to run “python app.py” and browse to localhost:8501 (Streamlit’s default port)
- Requires OPENAI_API_KEY in .env for embeddings and completion calls

##### Main components
a. User (Web Browser)
b. Frontend UI
– Serves static files (HTML, JS, CSS)
– Handles file upload & chat interface
c. Backend Web Server (app.py inside Docker container)
- PDF Processor
– Accepts PDF → text extraction & chunking
- Embedding Service
– Calls OpenAI Embeddings API for each chunk
- Vector Store Manager
– Creates Faiss index, saves to user_sessions/{id}/faiss_index/
– Retrieves top-k vectors at query time
- RAG QA Engine
– Composes prompt with retrieved chunks + user question
– Calls OpenAI ChatCompletions API → generates answer
- Session Manager
– Generates session UUIDs
– Persists chat_history.json
d. Local Storage
– user_sessions directory on host
– Persists vector index files and chat history per session
e. External Service
– OpenAI API (Embeddings & ChatCompletions)
f. Containerization
– Dockerfile builds image, injecting ENV var OPENAI_API_KEY

##### Relationships and data flows
1) User uploads PDF via Frontend → HTTP POST to Backend
2) Backend extracts text, chunks, calls Embedding Service → stores vectors in Local Storage
3) Frontend sends user question → Backend Session Manager loads chat_history + Faiss index
4) RAG QA Engine retrieves relevant chunks → calls OpenAI Completion → returns answer
5) Backend appends question/answer to chat_history.json → Frontend displays with source references

##### Architectural patterns & design principles
- RAG (Retrieval-Augmented Generation) pipeline
- Single-page frontend & RESTful or WebSocket backend
- Per-session isolation via directory structure
- Layered separation: Presentation (UI), Application Logic (app.py), Persistence (file system), External API

- User / Browser (e.g., blue)
- Frontend components (e.g., green)
- Backend services (e.g., orange)
- Upload flow (Browser → Backend)
- Embedding & completion calls (Backend → OpenAI)
- Persistence interactions (Backend ↔ Local FS)
- Response flow (Backend → Browser)
- Protocols (HTTP, file I/O, REST API calls)

## Diagram (Mermaid)

```mermaid
flowchart TD
    %% User Layer
    User["User (Browser)"]:::user

    %% Frontend Layer
    subgraph "Frontend UI" 
        direction TB
        IndexHTML["index.html"]:::frontend
        AppJS["app.js"]:::frontend
        StylesCSS["styles.css"]:::frontend
    end

    %% Backend Layer within Docker
    subgraph "Backend (Docker Container)" 
        direction TB
        BackendAPI["app.py\n(RAG API & App Logic)"]:::backend
        PDFProcessor["PDF Processor\n(text extraction & chunking)"]:::backend
        EmbeddingService["Embedding Service\n(OpenAI Embeddings)"]:::backend
        VectorStore["Vector Store Manager\n(Faiss index)"]:::backend
        RAGEngine["RAG QA Engine\n(prompt comp & completions)"]:::backend
        SessionMgr["Session Manager\n(UUID & chat history)"]:::backend
    end

    %% Storage Layer
    subgraph "Local Storage" 
        direction TB
        SessionsDir[(user_sessions/)]:::storage
        ChatHistory["chat_history.json"]:::storage
        FaissIndex1["index.faiss"]:::storage
        FaissIndex2["index.pkl"]:::storage
    end

    %% External Layer
    OpenAI["OpenAI API\n(Embeddings & ChatCompletions)"]:::external

    %% Infrastructure Files
    DockerfileNode["Dockerfile"]:::infra
    Dockerignore[".dockerignore"]:::infra
    Requirements["requirements.txt"]:::infra
    README["README.md"]:::infra

    %% Connections
    User -->|"loads UI"| IndexHTML
    User --> AppJS
    User --> StylesCSS

    AppJS -->|"HTTP POST: upload PDF"| PDFProcessor
    PDFProcessor -->|"chunks text"| EmbeddingService
    EmbeddingService -->|"REST API"| OpenAI
    EmbeddingService -->|"store vectors"| VectorStore

    AppJS -->|"sends question"| RAGEngine
    RAGEngine -->|"retrieves vectors"| VectorStore
    RAGEngine -->|"REST API"| OpenAI
    RAGEngine --> SessionMgr
    SessionMgr -->|"update chat"| ChatHistory

    PDFProcessor --> SessionsDir
    EmbeddingService --> SessionsDir
    VectorStore --> SessionsDir
    SessionMgr --> SessionsDir

    SessionsDir --> ChatHistory
    SessionsDir --> FaissIndex1
    SessionsDir --> FaissIndex2

    DockerfileNode --> BackendAPI
    Dockerignore --> BackendAPI
    Requirements --> BackendAPI
    README --> BackendAPI

    %% Click Events
    click IndexHTML "https://github.com/mildsauce9/documentor/blob/main/static/index.html"
    click AppJS "https://github.com/mildsauce9/documentor/blob/main/static/app.js"
    click StylesCSS "https://github.com/mildsauce9/documentor/blob/main/static/styles.css"
    click BackendAPI "https://github.com/mildsauce9/documentor/blob/main/app.py"
    click PDFProcessor "https://github.com/mildsauce9/documentor/blob/main/app.py"
    click EmbeddingService "https://github.com/mildsauce9/documentor/blob/main/app.py"
    click RAGEngine "https://github.com/mildsauce9/documentor/blob/main/app.py"
    click SessionMgr "https://github.com/mildsauce9/documentor/blob/main/user_sessions/{session_uuid}/chat_history.json"
    click ChatHistory "https://github.com/mildsauce9/documentor/blob/main/user_sessions/{session_uuid}/chat_history.json"
    click FaissIndex1 "https://github.com/mildsauce9/documentor/blob/main/user_sessions/{session_uuid}/faiss_index/index.faiss"
    click FaissIndex2 "https://github.com/mildsauce9/documentor/blob/main/user_sessions/{session_uuid}/faiss_index/index.pkl"
    click SessionsDir "https://github.com/mildsauce9/documentor/tree/main/user_sessions/"
    click DockerfileNode "https://github.com/mildsauce9/documentor/tree/main/Dockerfile"
    click Dockerignore "https://github.com/mildsauce9/documentor/blob/main/.dockerignore"
    click Requirements "https://github.com/mildsauce9/documentor/blob/main/requirements.txt"
    click README "https://github.com/mildsauce9/documentor/blob/main/README.md"

    %% Styles
    classDef user fill:#AED6F1,stroke:#21618C,color:#1B4F72;
    classDef frontend fill:#A9DFBF,stroke:#196F3D,color:#145A32;
    classDef backend fill:#F5B041,stroke:#B9770E,color:#7E5109;
    classDef storage fill:#F9E79F,stroke:#B7950B,color:#7D6608;
    classDef external fill:#D2B4DE,stroke:#6C3483,color:#4A235A;
    classDef infra fill:#D5DBDB,stroke:#566573,color:#1C2833;
```
