# hamida-c/invoice-fraud-detection architecture

> Architecture diagram of the GitHub repository hamida-c/invoice-fraud-detection, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-03-04). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/hamida-c/invoice-fraud-detection
- Repository: https://github.com/hamida-c/invoice-fraud-detection

## Overview

##### Project type & purpose (from README + tree)
This is a **single-repo, Python-based ML/AI application** delivered as a **Streamlit web app** for **real-time invoice fraud/anomaly detection**. It is an end-to-end pipeline:

**Invoice image upload → OCR (EasyOCR) → field extraction (regex) → feature engineering → rule-based checks + autoencoder anomaly score → fraud risk + explanation in UI**

There is no “backend service” folder; the “system” is mostly a **local inference pipeline** packaged into `streamlit.py`, with trained artifacts stored in `/model` and `/feature cols and encoder`.

##### What the file structure implies architecturally

- `streamlit.py`
**Application entrypoint**. Contains UI layer and likely orchestrates the full inference pipeline.

- `main.ipynb`

- `dataset/…csv`
**Training/validation data** (offline). Not used at runtime except for development.

- `model/invoice_autoencoder.h5`
Trained **TensorFlow/Keras autoencoder** for anomaly detection (reconstruction error).

- `feature cols and encoder/feature_cols.pkl`
Defines the **exact feature schema/order** expected by the model. Critical for runtime correctness.

- `feature cols and encoder/scaler_invoice.pkl`
A **scaler/normalizer** (sklearn) used before feeding features into the autoencoder.

- `feature cols and encoder/threshold.txt`
The **decision threshold** for reconstruction error → “fraud/anomaly” classification.

This structure strongly suggests two major “subsystems”:
##### Offline model development/training (notebook + dataset → artifacts)
2) **Online inference & UI** (Streamlit + OCR + feature pipeline + artifacts → decision + explanation)

##### Offline Training & Artifact Generation (lifecycle)

This avoids mixing concerns and makes data/artifact provenance explicit.

##### User / Client
- **End User (Browser)**
Interacts with Streamlit UI.

##### Document Ingestion & Preprocessing
- **File Handler**
- Accepts uploaded file bytes
- **Image Preprocessing (PIL)**
- Convert to image, resize/denoise if applicable

##### OCR Layer
- **EasyOCR Engine**
- Input: preprocessed invoice image

##### Parsing / Field Extraction
- **Regex Field Extractor**
- Extracts: amounts, dates, vendor name, invoice id, etc.
- Output: structured fields (JSON-like dict)

##### Feature Engineering
- **Feature Builder (15+ features)**
- Financial features (e.g., totals, tax patterns, currency format)
- Textual features (e.g., vendor string patterns)
- Completeness/consistency checks (missing fields, date anomalies)

##### Feature Schema Alignment
- **Feature Column Selector (`feature_cols.pkl`)**
- Enforces exact order and presence of features
- Handles missing columns (fill defaults) if implemented

##### Scaling / Normalization
- **Scaler (`scaler_invoice.pkl`)**
- Transforms features to the scale used in training

##### ML Inference: Autoencoder
- **Keras Model (`invoice_autoencoder.h5`)**
- Produces reconstruction (or reconstruction error)
- **Reconstruction Error Calculator**
- Metric: MSE/MAE across features (whatever notebook defines)

##### Decision Logic (Hybrid)
- **Threshold Comparator (`threshold.txt`)**
- reconstruction_error > threshold => anomaly
- **Rule-Based Fraud Checks**
- Deterministic rules (e.g., suspicious date gaps, negative totals, etc.)
- **Risk Aggregator**
- Combines rule flags + anomaly score into final fraud risk label

##### Explainability / Output Formatter
- **Explanation Generator**
- “Triggered rules: …”
- “Reconstruction error exceeded threshold by …”
- **Response Payload**
- Fraud label, score(s), extracted fields, flagged reasons

##### Local Storage (Artifacts)
- `/model/invoice_autoencoder.h5`
- `/feature cols and encoder/*.pkl`
- `threshold.txt`

Even if it’s “just a notebook,” capture it explicitly because it explains where artifacts come from.

##### Data Source
- **CSV Dataset (`dataset/*.csv`)**

##### Training Pipeline (Notebook)
- **Training & Experimentation (`main.ipynb`)**
- Data cleaning
- Feature engineering definition
- Fit scaler
- Train autoencoder
- Select threshold (e.g., percentile of reconstruction error)

##### Outputs / Artifacts
- **Model Artifact**
- `invoice_autoencoder.h5`
- **Preprocessing Artifacts**
- `scaler_invoice.pkl`
- `feature_cols.pkl`
- **Decision Artifact**
- `threshold.txt`

User Upload → Preprocess → OCR → Regex Extraction → Feature Engineering → Scaling → Autoencoder → Error Score → Threshold/Rules → Fraud Decision → UI Output

Streamlit App → loads → Model (`.h5`)
Streamlit App → loads → Scaler (`.pkl`)
Streamlit App → loads → Feature Columns (`.pkl`)
Streamlit App → loads → Threshold (`.txt`)

##### External library boundaries (optional but useful)
OCR (EasyOCR) and ML (TensorFlow/Keras) as labeled subcomponents to clarify runtime dependencies.

##### Architectural patterns to highlight (specific to this repo)
- **Pipeline architecture**: sequential stages from ingestion to decision.
- **Hybrid detection**: deterministic **rule-based** + learned **anomaly detection** (autoencoder).
- **Artifact-driven inference**: runtime behavior is governed by versioned artifacts (model + scaler + schema + threshold).
- **Single-process web app**: Streamlit likely runs everything in-process (no separate API server).

- Streamlit (UI + orchestration)
- EasyOCR (OCR)
- Regex (field parsing)
- Pandas/NumPy (feature engineering)
- Scikit-learn (scaler + joblib)
- TensorFlow/Keras (autoencoder inference)
- PIL (image preprocessing)

##### What NOT to do (common inaccuracies)
- Don’t omit the **artifact files** (they are essential system components).
- Don’t collapse the detection logic into “model predicts fraud” (it’s explicitly **autoencoder + rules + threshold**).

## Diagram (Mermaid)

```mermaid
flowchart TD
  %% =======================
  %% RUNTIME / INFERENCE LANE
  %% =======================
  subgraph "Runtime: Streamlit Inference Pipeline (Single Process)"
    direction TB

    U["End User (Browser)"]:::external

    subgraph "Presentation & Orchestration"
      direction TB
      ST["Streamlit App (UI + Orchestration)"]:::ui
      UP["Upload Widget\n(invoice image/PDF bytes)"]:::runtime
      RV["Results View\n(score,label,fields,explanations)"]:::runtime
    end

    subgraph "Document Ingestion & Preprocessing"
      direction TB
      FH["File Handler\n(read bytes, validate type)"]:::runtime
      IP["Image Preprocessing (PIL)\nconvert/resize/denoise"]:::runtime
    end

    subgraph "OCR & Parsing"
      direction TB
      OCR["EasyOCR Engine\n(extract text + boxes/confidence)"]:::externalLib
      RX["Regex Field Extractor\n(amounts,dates,vendor,invoice id,...)"]:::runtime
      SF["Structured Fields\n(JSON-like dict)"]:::dataNode
    end

    subgraph "Feature Pipeline"
      direction TB
      FB["Feature Builder (15+ features)\nfinancial,textual,consistency"]:::runtime
      FA["Feature Schema Alignment\n(select/order/fill defaults)"]:::runtime
      SC["Scaling / Normalization\n(sklearn scaler transform)"]:::runtime
      FV["Feature Vector\n(ordered,scaled)"]:::dataNode
    end

    subgraph "Anomaly Detection (Hybrid)"
      direction TB
      AE["Autoencoder Inference\n(TensorFlow/Keras)"]:::externalLib
      RE["Reconstruction Error Calculator\n(MSE/MAE across features)"]:::runtime
      TH["Threshold Comparator\n(error > threshold?)"]:::runtime
      RB["Rule-Based Fraud Checks\n(deterministic rules)"]:::runtime
      RA["Risk Aggregator\n(combine rules + anomaly score)"]:::runtime
    end

    subgraph "Explainability & Response"
      direction TB
      EG["Explanation Generator\n(triggered rules + threshold margin)"]:::runtime
      RP["Response Payload\n(label,scores,fields,reasons)"]:::dataNode
    end

    %% Data flow (solid)
    U -->|"interacts"| ST
    ST -->|"accept upload"| UP
    UP -->|"provide bytes"| FH
    FH -->|"decode/convert"| IP
    IP -->|"image -> text"| OCR
    OCR -->|"raw text"| RX
    RX -->|"emit"| SF
    SF -->|"build features"| FB
    FB -->|"engineered features"| FA
    FA -->|"aligned features"| SC
    SC -->|"scaled vector"| FV
    FV -->|"infer reconstruction"| AE
    AE -->|"reconstruction output"| RE
    RE -->|"error score"| TH
    RE -->|"error score"| RA
    TH -->|"anomaly flag"| RA
    SF -->|"input fields"| RB
    RB -->|"rule flags"| RA
    RA -->|"final risk + reasons"| EG
    EG -->|"format"| RP
    RP -->|"render results"| RV
    RV -->|"display"| U
  end

  %% =======================
  %% OFFLINE / TRAINING LANE
  %% =======================
  subgraph "Offline: Training & Artifact Generation (Lifecycle)"
    direction TB

    DS["Training/Validation Dataset\n(CSV files)"]:::storage
    NB["Training & Experimentation Notebook\n(data clean, features, fit scaler,\ntrain autoencoder, choose threshold)"]:::offline

    subgraph "Generated Artifacts"
      direction TB
      M_H5["Autoencoder Model Artifact\n(invoice_autoencoder.h5)"]:::storage
      FC_PKL["Feature Schema Artifact\n(feature_cols.pkl)"]:::storage
      SC_PKL["Scaler Artifact\n(scaler_invoice.pkl)"]:::storage
      TH_TXT["Decision Threshold Artifact\n(threshold.txt)"]:::storage
    end

    DS -->|"load CSVs"| NB
    NB -->|"save model"| M_H5
    NB -->|"save feature schema"| FC_PKL
    NB -->|"save scaler"| SC_PKL
    NB -->|"save threshold"| TH_TXT
  end

  %% =======================
  %% ARTIFACT STORE (LOCAL FS)
  %% =======================
  subgraph "Local Artifact Store (Filesystem)"
    direction TB
    AS["Artifact Store\n(local project folders)"]:::storage
  end

  %% Artifacts live in local store
  AS -->|"contains"| M_H5
  AS -->|"contains"| FC_PKL
  AS -->|"contains"| SC_PKL
  AS -->|"contains"| TH_TXT

  %% Dependency/load (dashed)
  ST -.->|"load model"| M_H5
  ST -.->|"load feature columns"| FC_PKL
  ST -.->|"load scaler"| SC_PKL
  ST -.->|"load threshold"| TH_TXT

  %% =======================
  %% CLICK EVENTS (paths only)
  %% =======================
  click ST "https://github.com/hamida-c/invoice-fraud-detection/blob/master/streamlit.py"
  click NB "https://github.com/hamida-c/invoice-fraud-detection/blob/master/main.ipynb"
  click DS "https://github.com/hamida-c/invoice-fraud-detection/tree/master/dataset/"
  click M_H5 "https://github.com/hamida-c/invoice-fraud-detection/blob/master/model/invoice_autoencoder.h5"
  click FC_PKL "https://github.com/hamida-c/invoice-fraud-detection/blob/master/feature cols and encoder/feature_cols.pkl"
  click SC_PKL "https://github.com/hamida-c/invoice-fraud-detection/blob/master/feature cols and encoder/scaler_invoice.pkl"
  click TH_TXT "https://github.com/hamida-c/invoice-fraud-detection/blob/master/feature cols and encoder/threshold.txt"
  click AS "https://github.com/hamida-c/invoice-fraud-detection/tree/master/model/"
  click AS "https://github.com/hamida-c/invoice-fraud-detection/tree/master/feature cols and encoder/"

  %% =======================
  %% STYLES
  %% =======================
  classDef ui fill:#2E7DFF,stroke:#0B2E6B,color:#FFFFFF,stroke-width:1.5px
  classDef runtime fill:#6CB6FF,stroke:#0B2E6B,color:#06243E,stroke-width:1.2px
  classDef externalLib fill:#FFB74D,stroke:#8A4B00,color:#3B2200,stroke-width:1.2px
  classDef storage fill:#BDBDBD,stroke:#424242,color:#1F1F1F,stroke-width:1.2px
  classDef offline fill:#A5D6A7,stroke:#1B5E20,color:#0D2B12,stroke-width:1.2px
  classDef external fill:#26C6DA,stroke:#006064,color:#003238,stroke-width:1.2px
  classDef dataNode fill:#E3F2FD,stroke:#1565C0,color:#0D2A4A,stroke-width:1px,stroke-dasharray: 3 3
```
