# tirthkothari/preciseextract architecture

> Architecture diagram of the GitHub repository tirthkothari/preciseextract, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-06-08). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/tirthkothari/preciseextract
- Repository: https://github.com/tirthkothari/preciseextract

## Overview

##### Project type and purpose
- This is a full-stack web application (“Precise Extract”) for bulk extraction of tabular data from PDFs/images, storage in MySQL, natural-language querying via Gemini/ChatGPT, and Excel export.
- It uses a Flask “main” server for user interaction (signup/login, upload, query UI) and a FastAPI microservice for file ingestion, OCR/extraction, and storage.

##### File structure
- Top-level folders:
- Backend/Precise_extract – Flask app (routes.py, forms.py, templates/, static/)
- fastapi – FastAPI service (myapp.py, UPLOADED_FILES/, Extract/PaddleOCR/)
- DATASET – sample JSON/images
- precise_extract.sql – database schema
- requirements.txt, LICENSE, README.md
- Flask server entrypoint: run.py
- FastAPI entrypoint: myapp.py (serving on port 8000, JWT/AES authentication)
- Templates & static assets define the frontend UI pages (login, upload, browse, query, export).
- “UPLOADED_FILES” holds user uploads; “Extract/PaddleOCR” handles OCR logic.

##### README insights
- Workflow steps: User → File upload → FastAPI (OCR via PaddleOCR) → MySQL storage → Query (Flask UI calls Gemini/ChatGPT → SQL) → Export.
- Two backend services: Flask (port 5000) for UI and auth, FastAPI (port 8000) for ingestion and extraction.
- Uses MySQL, PaddleOCR, Gemini/ChatGPT API.

##### Main components & relationships
a. Frontend (Flask templates + static JS/CSS)
b. Flask backend
– Routes for signup/login, upload UI, query UI, Excel export
– Calls FastAPI endpoints for file ingestion (via HTTP + JWT token)
– Calls external API (Gemini/ChatGPT) for NL→SQL conversion
c. FastAPI microservice
– Receives file uploads, applies AES decryption/JWT auth
– Converts PDF→images, invokes PaddleOCR module to extract tables
– Writes extracted rows to MySQL
d. MySQL database
e. External services
– PaddleOCR (local library)
– Gemini/ChatGPT API for natural-language query translation
f. Storage
– UPLOADED_FILES folder (temporary file storage)
– DATASET folder (sample data for testing)

Data flows:
1) User → Flask UI → HTTP POST to FastAPI /upload
2) FastAPI → OCR → MySQL insert
3) User submits query → Flask → Gemini/ChatGPT → SQL → MySQL → results
4) User requests export → Flask → reads MySQL → generate Excel → frontend download

##### Architectural patterns & design principles
- Separation of concerns: UI/auth server (Flask) vs ingestion service (FastAPI)
- Microservice-style: FastAPI service dedicated to heavy OCR/extraction tasks
- Stateless HTTP communication with JWT/AES for securing file uploads
- Modular extraction logic under fastapi/Extract/PaddleOCR
- MVC-like structure in Flask (templates/views/forms/models)

- Data store details (MySQL schema: precise_extract.sql)
- Validate with the team that flows match actual implementation (e.g., template names, API endpoints).

## Diagram (Mermaid)

```mermaid
flowchart TD
    %% Frontend Layer
    Browser["User Browser"]:::frontend

    %% Backend Services
    subgraph "Flask UI & Auth Service (port 5000)"
        FlaskUI["Flask Service"]:::backend
        Routes["routes.py"]:::backend
        Forms["forms.py"]:::backend
        Templates["templates/"]:::backend
        Static["static/"]:::backend
        Run["run.py"]:::backend
    end

    subgraph "FastAPI OCR Service (port 8000)"
        FastAPI["FastAPI Service"]:::backend
        Myapp["myapp.py"]:::backend
        OCRMod["Extract/PaddleOCR/"]:::backend
        UploadDir["UPLOADED_FILES/"]:::storage
        ExampleFiles["Example_Files/"]:::storage
    end

    %% Database & Storage
    MySQL["MySQL Database"]:::db
    FileStorage["File Storage (Uploads)"]:::storage
    SampleData["Sample Data (JSON/Images)"]:::storage

    %% External Services
    Gemini["Gemini/ChatGPT API"]:::external

    %% Data Flows
    Browser -->|"HTTP/HTTPS"| FlaskUI
    FlaskUI -->|"POST /upload\nJWT/AES"| FastAPI
    FastAPI -->|"invoke"| OCRMod
    OCRMod -->|"extracted rows"| MySQL
    FastAPI -->|"store files"| FileStorage
    FlaskUI -->|"read/write"| FileStorage

    Browser -->|"Query via UI"| FlaskUI
    FlaskUI -->|"NL→API call"| Gemini
    Gemini -->|"SQL Query"| MySQL
    MySQL -->|"results"| FlaskUI
    FlaskUI -->|"Excel Export"| Browser

    SampleData -.->|"for testing"| FastAPI

    %% Click Events
    click FlaskUI "https://github.com/tirthkothari/preciseextract/tree/main/Backend/Precise_extract/"
    click Routes "https://github.com/tirthkothari/preciseextract/blob/main/Backend/Precise_extract/routes.py"
    click Forms "https://github.com/tirthkothari/preciseextract/blob/main/Backend/Precise_extract/forms.py"
    click Templates "https://github.com/tirthkothari/preciseextract/tree/main/Backend/Precise_extract/templates/"
    click Static "https://github.com/tirthkothari/preciseextract/tree/main/Backend/Precise_extract/static/"
    click Run "https://github.com/tirthkothari/preciseextract/blob/main/Backend/run.py"
    click Myapp "https://github.com/tirthkothari/preciseextract/blob/main/fastapi/myapp.py"
    click OCRMod "https://github.com/tirthkothari/preciseextract/tree/main/fastapi/Extract/PaddleOCR/"
    click UploadDir "https://github.com/tirthkothari/preciseextract/tree/main/fastapi/UPLOADED_FILES/"
    click ExampleFiles "https://github.com/tirthkothari/preciseextract/tree/main/fastapi/Example_Files/"
    click MySQL "https://github.com/tirthkothari/preciseextract/blob/main/precise_extract.sql"
    click SampleData "https://github.com/tirthkothari/preciseextract/tree/main/DATASET/"
    click require "https://github.com/tirthkothari/preciseextract/blob/main/requirements.txt"

    %% Styles
    classDef frontend fill:#cfe2ff,stroke:#0366d6,color:#0366d6;
    classDef backend fill:#d1e7dd,stroke:#0f5132,color:#0f5132;
    classDef db fill:#ffe5b4,stroke:#e09c25,color:#e09c25;
    classDef storage fill:#fff3cd,stroke:#ffecb5,color:#997404;
    classDef external fill:#e2cbf0,stroke:#6f42c1,color:#6f42c1;
```
