How shafqaatmalik/llm-based-invoice-ocr works
This is a web-based invoice OCR pipeline: users upload a PDF or image, choose paid vision-model extraction or open-source OCR, and receive aggregated, schema-checked JSON. The graph follows the documented multi-page conversion and extraction stages and the calls visible in the sampled source. The README documents both extraction modes; no persistent application store is described.
Main entry points: main.py FastAPI endpoint · together_api.py Together extractor · open_source_mode.py OCR heuristics · gradio_app.py Gradio interface
Components 14
User interface
- Gradio interface
src/frontend/gradio_app.py
Invoice pipeline
- FastAPI endpoint
src/backend/main.py - PDF conversion
src/backend/main.py - Page images
src/backend/main.py - Mode dispatch
src/backend/main.py - Page aggregation
src/backend/parser.py - Schema validation
src/backend/parser.py - Structured JSON
src/backend/main.py
Extraction modes
- Together extractor
src/backend/together_api.py - OCR heuristics
src/backend/open_source_mode.py
External services
- Together AI
- Tesseract OCR
- Poppler renderer
Other components
- Invoice user
Connections 18
- Invoice user → Gradio interface: uploads and selects
- Gradio interface → FastAPI endpoint: posts file and mode
- FastAPI endpoint → PDF conversion: converts PDFs
- PDF conversion → Poppler renderer: renders pages
- PDF conversion → Page images: writes page images
- FastAPI endpoint → Page images: copies image uploads
- FastAPI endpoint → Mode dispatch: dispatches pages
- Mode dispatch → Together extractor: paid mode
- Mode dispatch → OCR heuristics: open-source mode
- Together extractor → Page images: reads image
- Together extractor → Together AI: sends image request
- Together AI → Together extractor: returns model response
- OCR heuristics → Page images: reads image
- OCR heuristics → Tesseract OCR: runs OCR
- Mode dispatch → Page aggregation: aggregates page results
- Page aggregation → Schema validation: passes merged fields
- FastAPI endpoint → Structured JSON: returns validated JSON
- Structured JSON → Gradio interface: displays response