# erudite5297/invoice-processing-pipeline architecture

> Architecture diagram of the GitHub repository erudite5297/invoice-processing-pipeline, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-10-06). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/erudite5297/invoice-processing-pipeline
- Repository: https://github.com/erudite5297/invoice-processing-pipeline

## Overview

This repository implements a Spark-based invoice batch processor. Its principal workflow reads invoice and vendor CSVs, validates invoices, removes duplicates, enriches and filters records, aggregates payable totals, and writes result CSVs. The same batch entry point also runs SQL and RDD demonstrations. The model and utility files define a case class and a total-calculation helper, but the sampled batch code does not use them. The streaming module has no visible implementation in the supplied excerpt, so its behavior and connections are omitted.

Main entry points: [`src/main/scala/BatchProcessor.scala`](https://github.com/erudite5297/invoice-processing-pipeline/blob/main/src/main/scala/BatchProcessor.scala) (Invoice batch processor), [`data/input/invoices.csv`](https://github.com/erudite5297/invoice-processing-pipeline/blob/main/data/input/invoices.csv) (Invoice CSV), [`data/reference/vendors.csv`](https://github.com/erudite5297/invoice-processing-pipeline/blob/main/data/reference/vendors.csv) (Vendor CSV)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_batch["Batch processing"]
  node_processor["Invoice batch processor"]
  node_output_files["Result CSV files"]
end

subgraph group_inputs["Input data"]
  node_invoice_csv["Invoice CSV<br/>[invoices.csv]"]
  node_vendor_csv["Vendor CSV<br/>[vendors.csv]"]
end

subgraph group_support["Domain support"]
  node_invoice_model["Invoice model<br/>[Models.scala]"]
  node_utils["Total helper<br/>[Utils.scala]"]
end

node_operator(("Batch operator"))

node_operator -->|"runs"| node_processor
node_processor -->|"reads"| node_invoice_csv
node_processor -->|"reads"| node_vendor_csv
node_processor -->|"writes"| node_output_files

click node_processor "https://github.com/erudite5297/invoice-processing-pipeline/blob/main/src/main/scala/BatchProcessor.scala"
click node_invoice_csv "https://github.com/erudite5297/invoice-processing-pipeline/blob/main/data/input/invoices.csv"
click node_vendor_csv "https://github.com/erudite5297/invoice-processing-pipeline/blob/main/data/reference/vendors.csv"
click node_invoice_model "https://github.com/erudite5297/invoice-processing-pipeline/blob/main/src/main/scala/Models.scala"
click node_utils "https://github.com/erudite5297/invoice-processing-pipeline/blob/main/src/main/scala/Utils.scala"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_processor,node_output_files toneBlue
class node_invoice_csv,node_vendor_csv toneAmber
class node_invoice_model,node_utils toneMint
class node_operator toneIndigo
```

## Components

### Batch processing

- [Invoice batch processor](https://github.com/erudite5297/invoice-processing-pipeline/blob/main/src/main/scala/BatchProcessor.scala): `src/main/scala/BatchProcessor.scala`
- **Result CSV files**

### Input data

- [Invoice CSV](https://github.com/erudite5297/invoice-processing-pipeline/blob/main/data/input/invoices.csv): `data/input/invoices.csv`
- [Vendor CSV](https://github.com/erudite5297/invoice-processing-pipeline/blob/main/data/reference/vendors.csv): `data/reference/vendors.csv`

### Domain support

- [Invoice model](https://github.com/erudite5297/invoice-processing-pipeline/blob/main/src/main/scala/Models.scala): `src/main/scala/Models.scala`
- [Total helper](https://github.com/erudite5297/invoice-processing-pipeline/blob/main/src/main/scala/Utils.scala): `src/main/scala/Utils.scala`

### Other components

- **Batch operator**

## Connections

- Batch operator → Invoice batch processor: runs (evidence: `src/main/scala/BatchProcessor.scala`)
- Invoice batch processor → Invoice CSV: reads (evidence: `src/main/scala/BatchProcessor.scala`)
- Invoice batch processor → Vendor CSV: reads (evidence: `src/main/scala/BatchProcessor.scala`)
- Invoice batch processor → Result CSV files: writes (evidence: `src/main/scala/BatchProcessor.scala`)
