How yash-meshram/table-extractor works
Project type and purpose
• This is a Streamlit-driven PDF table extraction tool. It combines a lightweight web UI (Streamlit) with backend modules that perform OCR, AI-based table detection, and Excel output generation.
• Its goal is to let users upload multi-page PDF documents, automatically detect tables via OCR + Google’s Gemini model, and download a structured Excel workbook (one sheet per PDF page).
File structure
• Top-level:
• app/
– main.py: orchestrates the Streamlit UI and kicks off extraction steps
– extract_text.py: wraps Tesseract OCR to convert PDF→images→raw text
– extract_table.py: calls the Gemini API to identify table boundaries, columns, and cell data
• data/
– Temp/: staging area for uploaded PDFs (and intermediate images)
– original PDFs (0001-0250.pdf) for testing
• output_table_0001-0250.pdf.xlsx: example Excel output
• .txt, LICENSE, README.md for documentation
• No database— all storage is file-based. Configuration is minimal (no Dockerfile or deployment scripts present).
Key insights from README
• Streamlit for UI
• Tesseract OCR for image-to-text
• Google Gemini via Google Cloud API for table structure recognition
• pandas / openpyxl for Excel generation
• pdfplumber / Pillow for PDF→image conversion
Main components and their interactions
a. User Interface (Streamlit front end)
– File uploader widget
– “Submit” button triggers the extraction pipeline
– Download link for resulting Excel workbook
b. Ingestion & Storage
– Accepts PDF, writes to data/Temp
– Optionally uses pdfplumber or Pillow to convert PDF pages to images
c. OCR Module
– extract_text.py uses pytesseract to convert each page image → raw text & coordinates
d. AI Table Detection
– extract_table.py packages OCR output and sends it to the Gemini API
– Receives structured table metadata (row/column boundaries, cell content)
e. Data Processing & Excel Generation
– pandas assembles a DataFrame per page
– openpyxl writes one sheet per page, preserving table formatting
f. Output & Cleanup
– Saves final .xlsx in the project root
– Optionally deletes temp files
Architectural patterns and design principles
• Layered design: UI → Ingestion → OCR → AI inference → Transformation → Output
• Single-page web app model (Streamlit)— all front-to-back calls occur in a single Python process
• File-based staging instead of a DB for simplicity
• External service integration (Google Cloud Gemini API) encapsulated in a dedicated module
• Separation of concerns:
• extract_text.py (OCR only)
• extract_table.py (AI + parsing)