How aum8/bank-statement_extraction works
Project Type and Purpose
• This is a Python-based *data-extraction tool* that reads bank-statement PDFs and outputs structured JSON.
• It exposes two “front ends”: a terminal/CLI script (extractor.py) and a web API (fast_api.py using FastAPI).
• Under the hood it uses OCR (Tesseract) plus optional LLM/ML back-ends (local model API or OpenRouter/Gemini).
A. Interfaces
• CLI Interface (extractor.py)
• HTTP API (FastAPI app in fast_api.py) + Swagger UI
B. Core Extraction Engine
• PDF loader / preprocessor
• OCR processor (Tesseract)
• Header parser (IFSC, account no.)
• Table parser (transactions)
• JSON builder
C. External Services / Models
• Local OCR binary (Tesseract OCR)
• Optional local ML model endpoint (LOCAL_MODEL_API_BASE)
• Open-source model via OpenRouter/Gemini API
D. Configuration & Environment
• .env variables (API keys, local model URL, Tesseract path)
E. Outputs
• JSON file (output.json) for CLI
• JSON HTTP response for API
Data Flow and Relationships
• CLI or HTTP client → PDF input
• Pass PDF to Core Extraction Engine
• Preprocess → OCR → parsers → JSON builder
• Core Engine may call either:
• Local Tesseract binary for OCR
• Local model API or OpenRouter/Gemini for advanced extraction tasks
• Engine returns JSON to:
• CLI (writes output.json)
• FastAPI (returns HTTP response)
Architectural Patterns & Design Principles
• Separation of Concerns: interfaces (CLI vs. HTTP) are thin wrappers over a shared core extraction module.
• Pluggable Back-Ends: OCR/ML components abstracted behind environment-configurable endpoints.
• Single-Process Service: no separate microservices—both modes run in the same Python process but could be split if scaling needed.
Technologies and Frameworks
• Python 3.x
• FastAPI (HTTP server, swagger docs)
• Tesseract OCR (via pytesseract or similar)
• PDF parsing library (e.g., pdfplumber or PyPDF2)
• HTTP client for external model APIs (requests or httpx)
• Uvicorn (ASGI server)
3) Inside Extraction Engine, subdivide into:
• PDF Preprocessor → OCR Processor → Data Parsers → JSON Builder