# yash-meshram/table-extractor architecture

> Architecture diagram of the GitHub repository yash-meshram/table-extractor, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-06-13). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/yash-meshram/table-extractor
- Repository: https://github.com/yash-meshram/table-extractor

## Overview

##### Project type and purpose
- This is a Streamlit-driven PDF table extraction tool. It combines a lightweight web UI (Streamlit) with backend modules that perform OCR, AI-based table detection, and Excel output generation.
- Its goal is to let users upload multi-page PDF documents, automatically detect tables via OCR + Google’s Gemini model, and download a structured Excel workbook (one sheet per PDF page).

##### File structure
- Top-level:
- app/
– main.py: orchestrates the Streamlit UI and kicks off extraction steps
– extract_text.py: wraps Tesseract OCR to convert PDF→images→raw text
– extract_table.py: calls the Gemini API to identify table boundaries, columns, and cell data
- data/
– Temp/: staging area for uploaded PDFs (and intermediate images)
– original PDFs (0001-0250.pdf) for testing
- output_table_0001-0250.pdf.xlsx: example Excel output
- .txt, LICENSE, README.md for documentation
- No database— all storage is file-based. Configuration is minimal (no Dockerfile or deployment scripts present).

##### Key insights from README
- Streamlit for UI
- Tesseract OCR for image-to-text
- Google Gemini via Google Cloud API for table structure recognition
- pandas / openpyxl for Excel generation
- pdfplumber / Pillow for PDF→image conversion

##### Main components and their interactions
a. User Interface (Streamlit front end)
– File uploader widget
– “Submit” button triggers the extraction pipeline
– Download link for resulting Excel workbook
b. Ingestion & Storage
– Accepts PDF, writes to data/Temp
– Optionally uses pdfplumber or Pillow to convert PDF pages to images
c. OCR Module
– extract_text.py uses pytesseract to convert each page image → raw text & coordinates
d. AI Table Detection
– extract_table.py packages OCR output and sends it to the Gemini API
– Receives structured table metadata (row/column boundaries, cell content)
e. Data Processing & Excel Generation
– pandas assembles a DataFrame per page
– openpyxl writes one sheet per page, preserving table formatting
f. Output & Cleanup
– Saves final .xlsx in the project root
– Optionally deletes temp files

##### Architectural patterns and design principles
- Layered design: UI → Ingestion → OCR → AI inference → Transformation → Output
- Single-page web app model (Streamlit)— all front-to-back calls occur in a single Python process
- File-based staging instead of a DB for simplicity
- External service integration (Google Cloud Gemini API) encapsulated in a dedicated module
- Separation of concerns:
- extract_text.py (OCR only)
- extract_table.py (AI + parsing)

## Diagram (Mermaid)

```mermaid
flowchart TB
    subgraph "Python Process"
        direction TB
        UI["Streamlit Front End"]:::inproc
        subgraph "PDF Processing"
            direction TB
            Conv["PDF→Image Converter\n(pdfplumber/Pillow)"]:::inproc
            OCR["OCR Module\n(extract_text.py)"]:::inproc
        end
        AI["AI Table Detection\n(extract_table.py)"]:::inproc
        DataProc["Data Processing & Excel Generation\n(pandas/openpyxl)"]:::inproc
    end

    Temp[(PDF Staging\ndata/Temp)]:::storage
    OutputFile[(Excel Output\n.xlsx)]:::storage
    Cloud("Google Cloud Gemini API"):::external

    UI -->|"Upload PDF"| Temp
    Temp -->|"PDF file"| Conv
    Conv -->|"Page images"| OCR
    OCR -->|"OCR JSON"| AI
    AI -->|"table metadata"| DataProc
    DataProc -->|".xlsx file"| OutputFile
    OutputFile -->|"Download link"| UI

    AI -->|"API request"| Cloud
    Cloud -->|"table metadata"| AI

    click UI "https://github.com/yash-meshram/table-extractor/blob/main/app/main.py"
    click Temp "https://github.com/yash-meshram/table-extractor/tree/main/data/Temp/"
    click OCR "https://github.com/yash-meshram/table-extractor/blob/main/app/extract_text.py"
    click AI "https://github.com/yash-meshram/table-extractor/blob/main/app/extract_table.py"
    click DataProc "https://github.com/yash-meshram/table-extractor/blob/main/app/main.py"
    click OutputFile "https://github.com/yash-meshram/table-extractor/blob/main/output_table_0001-0250.pdf.xlsx"

    classDef inproc fill:#D6EAF8,stroke:#1B4F72,color:#1B2631
    classDef storage fill:#D5F5E3,stroke:#145A32,color:#0B5345
    classDef external fill:#FAD7A0,stroke:#B9770E,color:#7E5109
```
