How lianeheidemann/parallel-image-pipeline works
This repository is an experimental image-processing pipeline comparing sequential and parallel execution across local Python, browser Web Workers, and AWS EC2. The main workflow is to generate or supply images, apply the same grayscale/blur/Sobel transform, save PNG results, then benchmark execution and verify that sequential and parallel outputs match. The README documents the benchmark methodology and browser/AWS capabilities; some of those implementations were not sampled, so their internal wiring is shown only where documented.
Main entry points: sequential.py Sequential runner · parallel.py Process pool · benchmark.py Benchmark · server.py Read-only panel
Components 20
Python pipeline
- Dataset generator
local/generate_dataset.py - Input images
dataset - Sequential runner
local/sequential.py - Process pool
local/parallel.py - Image processor
local/image_processor.py - Shared file utilities
local/common.py - PNG outputs
output
Evaluation
- Benchmark
local/benchmark.py - SHA-256 verifier
local/verify.py - CSV results
results
Browser pipeline
- Web interface
web/index.html - Web application
web/assets/app.js - Worker coordinator
web/assets/runner.js - Web Worker
web/assets/worker.js - Pixel processor
web/assets/processor.js - Result renderer
web/assets/render.js
AWS panel
- Read-only panel
aws/server.py - EC2 metadata
Other components
- Experimenter
- Browser user
Connections 31
- Experimenter → Dataset generator: generates dataset
- Dataset generator → Input images: writes images
- Experimenter → Sequential runner: runs
- Experimenter → Process pool: runs
- Sequential runner → Shared file utilities: lists and prepares
- Process pool → Shared file utilities: lists and prepares
- Sequential runner → Input images: reads images
- Process pool → Input images: reads images
- Sequential runner → Image processor: processes each image
- Process pool → Image processor: processes each image
- Sequential runner → PNG outputs: writes PNGs
- Process pool → PNG outputs: writes PNGs
- Sequential runner → CSV results: writes per-image CSV
- Process pool → CSV results: writes per-image CSV
- Experimenter → Benchmark: runs comparison
- Benchmark → Sequential runner: measures sequential
- Benchmark → Process pool: measures parallel
- Benchmark → SHA-256 verifier: checks correctness
- SHA-256 verifier → PNG outputs: compares PNGs
- Benchmark → CSV results: records results
- Browser user → Web interface: opens
- Web interface → Web application: loads interface
- Web application → Worker coordinator: dispatches work
- Worker coordinator → Web Worker: assigns image bands
- Web Worker → Pixel processor: processes pixels
- Web Worker → Worker coordinator: returns band results
- Web application → Result renderer: displays results
- Experimenter → Read-only panel: requests panel
- Read-only panel → CSV results: reads benchmark CSV
- Read-only panel → Input images: summarizes images
- Read-only panel → EC2 metadata: queries instance details