How jhonvilcarana/facturia_proyect works
Project Type and Purpose
• FacturIA is a full-stack, data-processing application that automates invoice ingestion, extraction, storage, and visualization. It comprises:
• A document ingestion and processing pipeline (PDFs + images → structured data via OCR and AI).
• A backend REST API (Flask) exposing processed invoices.
• A relational database (PostgreSQL) for data persistence.
• An analytical dashboard (Power BI) for reporting.
File Structure Analysis
• Top-level scripts:
• main.py – orchestrates the invoice scanning, OCR (Tesseract), PDF text extraction (PyMuPDF), AI prompts, currency conversion, and DB insertion.
• funciones.py – utility functions for parsing, OCR, data normalization, error handling.
• prompt.py – houses the OpenAI GPT-4o-mini prompt templates and parsing logic.
• api_facturas.py – Flask app exposing GET /facturas endpoint.
• Folders:
• facturas/YYYY-MM/ – input folder for raw invoice files (PDF, JPG, PNG, TIFF, BMP).
• Dashboard_PowerBI/ – Power BI .pbix and resources for visualization.
• Config & Dependency files:
• .env.example – environment variables for OpenAI API key and DATABASE_URL.
• requirements.txt, entorno.yml – Python dependencies.
a. Ingestion Layer
• Invoice Storage (Local FS – facturas/ folder)
• File Watcher or Batch Script (main.py)
b. Processing Layer
• PDF Text Extractor (PyMuPDF)
• OCR Engine (Tesseract via pytesseract + Pillow)
• AI Processor (OpenAI GPT-4o-mini via prompt.py)
• Currency Converter & Data Cleaner (funciones.py)
c. Persistence Layer
• PostgreSQL DB (SQLAlchemy + psycopg2)
• DB Schema (invoices table: id, fecha, proveedor, importe, concepto, moneda)
d. API Layer
• Flask App (api_facturas.py)
• Endpoint: GET /facturas → returns JSON of processed records
e. Visualization Layer
• Power BI Dashboard (FacturIA.pbix)
• Connects to PostgreSQL to render charts and KPIs
f. External Services
• OpenAI API
• Tesseract Engine
Relationships:
• main.py reads files → passes to PyMuPDF or pytesseract → structured output → prompt.py enriches via OpenAI → funciones.py normalizes data → inserts into PostgreSQL.
• api_facturas.py queries PostgreSQL → returns JSON to clients.
• Power BI connects to PostgreSQL → fetches tables → visualizes data.
Architectural Patterns & Principles
• Layered Architecture: clear separation between ingestion, processing, persistence, API, and visualization.
• Single Responsibility: each script/folder handles a specific concern.
• Configuration via Env Vars: decoupled secrets and endpoints.
• Batch vs. Real-time: invoice folder polling (batch) vs. REST API (real-time).
• Extensibility: easily add new file types, languages, or AI models.