# lianeheidemann/parallel-image-pipeline architecture

> Architecture diagram of the GitHub repository lianeheidemann/parallel-image-pipeline, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-09-25). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/lianeheidemann/parallel-image-pipeline
- Repository: https://github.com/lianeheidemann/parallel-image-pipeline

## Overview

This repository is an experimental image-processing pipeline comparing sequential and parallel execution across local Python, browser Web Workers, and AWS EC2. The main workflow is to generate or supply images, apply the same grayscale/blur/Sobel transform, save PNG results, then benchmark execution and verify that sequential and parallel outputs match. The README documents the benchmark methodology and browser/AWS capabilities; some of those implementations were not sampled, so their internal wiring is shown only where documented.

Main entry points: [`local/sequential.py`](https://github.com/lianeheidemann/parallel-image-pipeline/blob/main/local/sequential.py) (Sequential runner), [`local/parallel.py`](https://github.com/lianeheidemann/parallel-image-pipeline/blob/main/local/parallel.py) (Process pool), [`local/benchmark.py`](https://github.com/lianeheidemann/parallel-image-pipeline/blob/main/local/benchmark.py) (Benchmark), [`aws/server.py`](https://github.com/lianeheidemann/parallel-image-pipeline/blob/main/aws/server.py) (Read-only panel)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_python["Python pipeline"]
  node_datasetgen["Dataset generator"]
  node_dataset["Input images"]
  node_sequential["Sequential runner<br/>[sequential.py]"]
  node_parallel["Process pool<br/>[parallel.py]"]
  node_processor["Image processor<br/>[image_processor.py]"]
  node_common["Shared file utilities<br/>[common.py]"]
  node_outputs["PNG outputs"]
end

subgraph group_evaluation["Evaluation"]
  node_benchmark["Benchmark<br/>[benchmark.py]"]
  node_verifier["SHA-256 verifier<br/>[verify.py]"]
  node_reports["CSV results"]
end

subgraph group_browser["Browser pipeline"]
  node_webpage["Web interface<br/>[index.html]"]
  node_app["Web application<br/>[app.js]"]
  node_runner["Worker coordinator<br/>[runner.js]"]
  node_worker["Web Worker<br/>[worker.js]"]
  node_webprocessor["Pixel processor<br/>[processor.js]"]
  node_renderer["Result renderer<br/>[render.js]"]
end

subgraph group_aws["AWS panel"]
  node_awsserver["Read-only panel<br/>[server.py]"]
  node_metadata["EC2 metadata"]
end

node_user(("Experimenter"))
node_browser(("Browser user"))

node_user -->|"generates dataset"| node_datasetgen
node_datasetgen -->|"writes images"| node_dataset
node_user -->|"runs"| node_sequential
node_user -->|"runs"| node_parallel
node_sequential -->|"lists and prepares"| node_common
node_parallel -->|"lists and prepares"| node_common
node_sequential -->|"reads images"| node_dataset
node_parallel -->|"reads images"| node_dataset
node_sequential -->|"processes each image"| node_processor
node_parallel -->|"processes each image"| node_processor
node_sequential -->|"writes PNGs"| node_outputs
node_parallel -->|"writes PNGs"| node_outputs
node_sequential -->|"writes per-image CSV"| node_reports
node_parallel -->|"writes per-image CSV"| node_reports
node_user -->|"runs comparison"| node_benchmark
node_benchmark -->|"measures sequential"| node_sequential
node_benchmark -->|"measures parallel"| node_parallel
node_benchmark -->|"checks correctness"| node_verifier
node_verifier -->|"compares PNGs"| node_outputs
node_benchmark -->|"records results"| node_reports
node_browser -->|"opens"| node_webpage
node_webpage -->|"loads interface"| node_app
node_app -->|"dispatches work"| node_runner
node_runner -->|"assigns image bands"| node_worker
node_worker -->|"processes pixels"| node_webprocessor
node_worker -->|"returns band results"| node_runner
node_app -->|"displays results"| node_renderer
node_user -.->|"requests panel"| node_awsserver
node_awsserver -->|"reads benchmark CSV"| node_reports
node_awsserver -->|"summarizes images"| node_dataset
node_awsserver -.->|"queries instance details"| node_metadata

click node_datasetgen "https://github.com/lianeheidemann/parallel-image-pipeline/blob/main/local/generate_dataset.py"
click node_dataset "https://github.com/lianeheidemann/parallel-image-pipeline/tree/main/dataset"
click node_sequential "https://github.com/lianeheidemann/parallel-image-pipeline/blob/main/local/sequential.py"
click node_parallel "https://github.com/lianeheidemann/parallel-image-pipeline/blob/main/local/parallel.py"
click node_processor "https://github.com/lianeheidemann/parallel-image-pipeline/blob/main/local/image_processor.py"
click node_common "https://github.com/lianeheidemann/parallel-image-pipeline/blob/main/local/common.py"
click node_outputs "https://github.com/lianeheidemann/parallel-image-pipeline/tree/main/output"
click node_benchmark "https://github.com/lianeheidemann/parallel-image-pipeline/blob/main/local/benchmark.py"
click node_verifier "https://github.com/lianeheidemann/parallel-image-pipeline/blob/main/local/verify.py"
click node_reports "https://github.com/lianeheidemann/parallel-image-pipeline/tree/main/results"
click node_webpage "https://github.com/lianeheidemann/parallel-image-pipeline/blob/main/web/index.html"
click node_app "https://github.com/lianeheidemann/parallel-image-pipeline/blob/main/web/assets/app.js"
click node_runner "https://github.com/lianeheidemann/parallel-image-pipeline/blob/main/web/assets/runner.js"
click node_worker "https://github.com/lianeheidemann/parallel-image-pipeline/blob/main/web/assets/worker.js"
click node_webprocessor "https://github.com/lianeheidemann/parallel-image-pipeline/blob/main/web/assets/processor.js"
click node_renderer "https://github.com/lianeheidemann/parallel-image-pipeline/blob/main/web/assets/render.js"
click node_awsserver "https://github.com/lianeheidemann/parallel-image-pipeline/blob/main/aws/server.py"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_datasetgen,node_dataset,node_sequential,node_parallel,node_processor,node_common,node_outputs,node_browser toneBlue
class node_benchmark,node_verifier,node_reports toneAmber
class node_webpage,node_app,node_runner,node_worker,node_webprocessor,node_renderer toneMint
class node_awsserver,node_metadata toneRose
class node_user toneIndigo
```

## Components

### Python pipeline

- [Dataset generator](https://github.com/lianeheidemann/parallel-image-pipeline/blob/main/local/generate_dataset.py): `local/generate_dataset.py`
- [Input images](https://github.com/lianeheidemann/parallel-image-pipeline/tree/main/dataset): `dataset`
- [Sequential runner](https://github.com/lianeheidemann/parallel-image-pipeline/blob/main/local/sequential.py): `local/sequential.py`
- [Process pool](https://github.com/lianeheidemann/parallel-image-pipeline/blob/main/local/parallel.py): `local/parallel.py`
- [Image processor](https://github.com/lianeheidemann/parallel-image-pipeline/blob/main/local/image_processor.py): `local/image_processor.py`
- [Shared file utilities](https://github.com/lianeheidemann/parallel-image-pipeline/blob/main/local/common.py): `local/common.py`
- [PNG outputs](https://github.com/lianeheidemann/parallel-image-pipeline/tree/main/output): `output`

### Evaluation

- [Benchmark](https://github.com/lianeheidemann/parallel-image-pipeline/blob/main/local/benchmark.py): `local/benchmark.py`
- [SHA-256 verifier](https://github.com/lianeheidemann/parallel-image-pipeline/blob/main/local/verify.py): `local/verify.py`
- [CSV results](https://github.com/lianeheidemann/parallel-image-pipeline/tree/main/results): `results`

### Browser pipeline

- [Web interface](https://github.com/lianeheidemann/parallel-image-pipeline/blob/main/web/index.html): `web/index.html`
- [Web application](https://github.com/lianeheidemann/parallel-image-pipeline/blob/main/web/assets/app.js): `web/assets/app.js`
- [Worker coordinator](https://github.com/lianeheidemann/parallel-image-pipeline/blob/main/web/assets/runner.js): `web/assets/runner.js`
- [Web Worker](https://github.com/lianeheidemann/parallel-image-pipeline/blob/main/web/assets/worker.js): `web/assets/worker.js`
- [Pixel processor](https://github.com/lianeheidemann/parallel-image-pipeline/blob/main/web/assets/processor.js): `web/assets/processor.js`
- [Result renderer](https://github.com/lianeheidemann/parallel-image-pipeline/blob/main/web/assets/render.js): `web/assets/render.js`

### AWS panel

- [Read-only panel](https://github.com/lianeheidemann/parallel-image-pipeline/blob/main/aws/server.py): `aws/server.py`
- **EC2 metadata**

### Other components

- **Experimenter**
- **Browser user**

## Connections

- Experimenter → Dataset generator: generates dataset
- Dataset generator → Input images: writes images
- Experimenter → Sequential runner: runs
- Experimenter → Process pool: runs
- Sequential runner → Shared file utilities: lists and prepares
- Process pool → Shared file utilities: lists and prepares
- Sequential runner → Input images: reads images
- Process pool → Input images: reads images
- Sequential runner → Image processor: processes each image
- Process pool → Image processor: processes each image
- Sequential runner → PNG outputs: writes PNGs
- Process pool → PNG outputs: writes PNGs
- Sequential runner → CSV results: writes per-image CSV
- Process pool → CSV results: writes per-image CSV
- Experimenter → Benchmark: runs comparison
- Benchmark → Sequential runner: measures sequential
- Benchmark → Process pool: measures parallel
- Benchmark → SHA-256 verifier: checks correctness
- SHA-256 verifier → PNG outputs: compares PNGs
- Benchmark → CSV results: records results
- Browser user → Web interface: opens
- Web interface → Web application: loads interface
- Web application → Worker coordinator: dispatches work
- Worker coordinator → Web Worker: assigns image bands
- Web Worker → Pixel processor: processes pixels
- Web Worker → Worker coordinator: returns band results
- Web application → Result renderer: displays results
- Experimenter → Read-only panel: requests panel
- Read-only panel → CSV results: reads benchmark CSV
- Read-only panel → Input images: summarizes images
- Read-only panel → EC2 metadata: queries instance details