# hemant-1010/automated-invoice-data-extractor architecture

> Architecture diagram of the GitHub repository hemant-1010/automated-invoice-data-extractor, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-06-24). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/hemant-1010/automated-invoice-data-extractor
- Repository: https://github.com/hemant-1010/automated-invoice-data-extractor

## Overview

1. This repository is a single-process Python batch pipeline, centered on `invoice_extractor.py`. There is no web server, database, or background worker architecture; the whole system runs locally from the command line and processes invoice PDFs one by one.

2. The input boundary is the `invoices/` folder. The script expects PDF files there, reads them from disk, and treats each PDF as one extraction unit. The sample PDFs at the repo root (`1.pdf`, `2.pdf`, `3.pdf`) appear to be test artifacts or examples, but the documented workflow is folder-based.

3. The first subsystem is PDF text acquisition. It uses `pdfplumber` to open each invoice and extract text across pages. This is the lowest-level dependency in the pipeline and is important because everything downstream depends on whether the PDF contains machine-readable text rather than scanned images.

4. The second subsystem is text normalization. After extraction, the raw text is cleaned by removing empty lines and extra whitespace. This preprocessing step creates a more stable input for the language model and reduces noise from multi-page invoice formatting.

5. The third subsystem is prompt construction. The script builds a tightly scoped extraction prompt that defines the 13 required invoice fields, date formatting rules, and null-handling behavior. This prompt is the main control boundary between unstructured document text and structured output.

6. The fourth subsystem is the OpenAI integration. The pipeline calls the OpenAI API using `gpt-4o-mini` with `temperature=0`, making the extraction deterministic and reducing hallucinated values. The model is used strictly as a structured information extractor, not as a conversational assistant.

7. The fifth subsystem is response parsing and validation. The model output is expected to be JSON, and the script parses it with `json.loads()`. The repository explicitly anticipates malformed responses, such as markdown code fences or non-JSON text, so parsing is a critical failure point in the architecture.

8. The sixth subsystem is presentation and persistence. Extracted data is printed in a human-readable console format and also written to JSON files in an `outputs/` directory. That output folder is generated as part of the pipeline and serves as the final artifact boundary for downstream use.

9. Operationally, the system is driven by environment configuration. The OpenAI API key is loaded from a `.env` file using `python-dotenv`, keeping credentials out of source code. This is the only meaningful external secret management concern in the repo.

10. The dependency stack is intentionally small: `pdfplumber` for PDF parsing, `openai` for LLM access, and `python-dotenv` for configuration loading. The repository’s architecture is therefore a thin orchestration layer around these libraries rather than a multi-service application.

11. Error handling is part of the pipeline boundary design. The README describes graceful handling for missing files, API failures, and JSON parsing issues, which means the script is expected to continue or fail cleanly at each stage instead of crashing silently.

12. From an architecture-diagram perspective, the data flow is linear: local PDF files in `invoices/` → text extraction and cleanup → prompt assembly → OpenAI model call → JSON parsing → console display and `outputs/` JSON files. The only external network dependency is the OpenAI API; everything else is local filesystem and Python runtime.

Main entry points: [`invoice_extractor.py`](https://github.com/hemant-1010/automated-invoice-data-extractor/blob/main/invoice_extractor.py) (Parse JSON), [`invoices`](https://github.com/hemant-1010/automated-invoice-data-extractor/tree/main/invoices) (Invoice PDFs)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_pipeline["Pipeline"]
  node_cli["CLI run<br/>entrypoint"]
  node_invoices[("Invoice PDFs<br/>input folder")]
  node_pdf_text["PDF text<br/>pdfplumber extract"]
  node_normalize["Clean text<br/>preprocess"]
  node_prompt["Build prompt<br/>prompt assembly"]
  node_openai["OpenAI call<br/>llm request"]
  node_response["Model JSON<br/>llm response"]
  node_parse{{"Parse JSON<br/>validation"}}
  node_present["Print data<br/>console output"]
  node_store[("Write JSON<br/>artifact output")]
  node_env(("API key<br/>config"))
end

node_cli -->|"process files"| node_invoices
node_invoices -->|"read PDF"| node_pdf_text
node_pdf_text -->|"clean text"| node_normalize
node_normalize -->|"assemble prompt"| node_prompt
node_prompt -->|"request"| node_openai
node_env -->|"authenticate"| node_openai
node_openai -->|"return data"| node_response
node_response -->|"decode"| node_parse
node_parse -->|"display"| node_present
node_parse -->|"persist"| node_store
node_parse -.->|"errors"| node_cli

click node_cli "https://github.com/hemant-1010/automated-invoice-data-extractor/blob/main/invoice_extractor.py"
click node_invoices "https://github.com/hemant-1010/automated-invoice-data-extractor/tree/main/invoices"
click node_pdf_text "https://github.com/hemant-1010/automated-invoice-data-extractor/blob/main/invoice_extractor.py"
click node_normalize "https://github.com/hemant-1010/automated-invoice-data-extractor/blob/main/invoice_extractor.py"
click node_prompt "https://github.com/hemant-1010/automated-invoice-data-extractor/blob/main/invoice_extractor.py"
click node_openai "https://github.com/hemant-1010/automated-invoice-data-extractor/blob/main/invoice_extractor.py"
click node_response "https://github.com/hemant-1010/automated-invoice-data-extractor/blob/main/invoice_extractor.py"
click node_parse "https://github.com/hemant-1010/automated-invoice-data-extractor/blob/main/invoice_extractor.py"
click node_present "https://github.com/hemant-1010/automated-invoice-data-extractor/blob/main/invoice_extractor.py"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_cli,node_invoices,node_pdf_text,node_normalize,node_prompt,node_openai,node_response,node_parse,node_present,node_store,node_env toneBlue
```

## Components

### Pipeline

Single-process invoice extraction flow driven by one Python script.

- [CLI run](https://github.com/hemant-1010/automated-invoice-data-extractor/blob/main/invoice_extractor.py): entrypoint · Local command-line execution of the batch pipeline. · `invoice_extractor.py`
- [Invoice PDFs](https://github.com/hemant-1010/automated-invoice-data-extractor/tree/main/invoices): input folder · Source PDFs processed one by one from the invoices directory. · `invoices`
- [PDF text](https://github.com/hemant-1010/automated-invoice-data-extractor/blob/main/invoice_extractor.py): pdfplumber extract · Reads machine-readable text across invoice pages. · `invoice_extractor.py`
- [Clean text](https://github.com/hemant-1010/automated-invoice-data-extractor/blob/main/invoice_extractor.py): preprocess · Removes empty lines and collapses extra whitespace. · `invoice_extractor.py`
- [Build prompt](https://github.com/hemant-1010/automated-invoice-data-extractor/blob/main/invoice_extractor.py): prompt assembly · Constructs the extraction prompt with required fields, date rules, and null handling. · `invoice_extractor.py`
- [OpenAI call](https://github.com/hemant-1010/automated-invoice-data-extractor/blob/main/invoice_extractor.py): llm request · Sends the prompt to gpt-4o-mini with deterministic settings. · `invoice_extractor.py`
- [Model JSON](https://github.com/hemant-1010/automated-invoice-data-extractor/blob/main/invoice_extractor.py): llm response · Expected structured invoice data returned by the model. · `invoice_extractor.py`
- [Parse JSON](https://github.com/hemant-1010/automated-invoice-data-extractor/blob/main/invoice_extractor.py): validation · Loads model output with json.loads and handles malformed responses. · `invoice_extractor.py`
- [Print data](https://github.com/hemant-1010/automated-invoice-data-extractor/blob/main/invoice_extractor.py): console output · Formats extracted invoice fields for human-readable display. · `invoice_extractor.py`
- **Write JSON**: artifact output · Persists extracted records to JSON files in the outputs artifact boundary.
- **API key**: config · Loads the OpenAI credential from .env via python-dotenv.

## Connections

- CLI run → Invoice PDFs: process files · Batch run starts by scanning invoice PDFs in the source folder.
- Invoice PDFs → PDF text: read PDF · Each invoice PDF is opened and text is extracted page by page.
- PDF text → Clean text: clean text · Raw text is normalized to remove blank lines and excess whitespace.
- Clean text → Build prompt: assemble prompt · Cleaned invoice text is inserted into the extraction prompt.
- Build prompt → OpenAI call: request · The prompt is sent to the OpenAI model for structured extraction.
- API key → OpenAI call: authenticate · The API key is loaded from environment configuration for the model call.
- OpenAI call → Model JSON: return data · The model responds with invoice fields in JSON form.
- Model JSON → Parse JSON: decode · The output is parsed as JSON and guarded against malformed formatting.
- Parse JSON → Print data: display · Validated invoice data is printed for the operator.
- Parse JSON → Write JSON: persist · Validated invoice data is written to output JSON artifacts.
- Parse JSON → CLI run: errors · Missing files, API failures, and parse issues are handled at the pipeline boundary.