# kaityo256/pycpp-parallel-pipeline architecture

> Architecture diagram of the GitHub repository kaityo256/pycpp-parallel-pipeline, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-11-09). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/kaityo256/pycpp-parallel-pipeline
- Repository: https://github.com/kaityo256/pycpp-parallel-pipeline

## Overview

This repository demonstrates an end-to-end parallel simulation workflow: Python expands a shared YAML configuration into independent job inputs, C++ runs each bond-percolation simulation, and Python aggregates the per-seed results for plotting. The documented `cps` scheduler distributes generated tasks, while gnuplot renders the summary. The scheduler and plotting implementation are not sampled, so those documented capabilities are included at the directory/script level; their internal relationships are limited to those stated in the README.

Main entry points: [`generate_inputs.py`](https://github.com/kaityo256/pycpp-parallel-pipeline/blob/main/generate_inputs.py) (Input generator), [`cpp/main.cpp`](https://github.com/kaityo256/pycpp-parallel-pipeline/blob/main/cpp/main.cpp) (Simulation entry), [`cps`](https://github.com/kaityo256/pycpp-parallel-pipeline/tree/main/cps) (CPS scheduler), [`analyze_results.py`](https://github.com/kaityo256/pycpp-parallel-pipeline/blob/main/analyze_results.py) (Result analyzer)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_prep["Job preparation"]
  node_config["YAML configuration<br/>[input.yaml]"]
  node_generator["Input generator<br/>[generate_inputs.py]"]
  node_job_inputs["Job input files"]
  node_task_list["Task list<br/>[generate_inputs.py]"]
end

subgraph group_compute["Simulation"]
  node_entry["Simulation entry<br/>[main.cpp]"]
  node_parser["Parameter parser"]
  node_engine["Percolation engine<br/>[percolation.hpp]"]
  node_job_results["Per-job results"]
end

subgraph group_parallel["Parallel execution"]
  node_scheduler["CPS scheduler"]
  node_mpi["MPI runtime"]
end

subgraph group_analysis["Results and plotting"]
  node_analyzer["Result analyzer<br/>[analyze_results.py]"]
  node_summary["Summary data"]
  node_plot_script["Plot definition<br/>[plot.plt]"]
  node_gnuplot["Gnuplot"]
  node_figure["Result figure"]
end

node_user(("Workflow user"))

node_user -->|"runs"| node_generator
node_config -->|"configures"| node_generator
node_generator -->|"writes inputs"| node_job_inputs
node_generator -->|"writes tasks"| node_task_list
node_task_list -->|"schedules commands"| node_scheduler
node_mpi -->|"runs scheduler"| node_scheduler
node_scheduler -->|"dispatches jobs"| node_entry
node_job_inputs -->|"provides input"| node_entry
node_entry -->|"parses parameters"| node_parser
node_entry -->|"runs task"| node_engine
node_engine -->|"writes result"| node_job_results
node_job_results -->|"provides results"| node_analyzer
node_config -->|"sets aggregation range"| node_analyzer
node_analyzer -->|"writes statistics"| node_summary
node_summary -->|"provides plot data"| node_gnuplot
node_plot_script -->|"configures plot"| node_gnuplot
node_gnuplot -->|"renders figure"| node_figure

click node_config "https://github.com/kaityo256/pycpp-parallel-pipeline/blob/main/input.yaml"
click node_generator "https://github.com/kaityo256/pycpp-parallel-pipeline/blob/main/generate_inputs.py"
click node_job_inputs "https://github.com/kaityo256/pycpp-parallel-pipeline/tree/main/output"
click node_task_list "https://github.com/kaityo256/pycpp-parallel-pipeline/blob/main/generate_inputs.py"
click node_entry "https://github.com/kaityo256/pycpp-parallel-pipeline/blob/main/cpp/main.cpp"
click node_parser "https://github.com/kaityo256/pycpp-parallel-pipeline/tree/main/cpp/param"
click node_engine "https://github.com/kaityo256/pycpp-parallel-pipeline/blob/main/cpp/percolation.hpp"
click node_job_results "https://github.com/kaityo256/pycpp-parallel-pipeline/tree/main/output"
click node_scheduler "https://github.com/kaityo256/pycpp-parallel-pipeline/tree/main/cps"
click node_analyzer "https://github.com/kaityo256/pycpp-parallel-pipeline/blob/main/analyze_results.py"
click node_summary "https://github.com/kaityo256/pycpp-parallel-pipeline/tree/main/output"
click node_plot_script "https://github.com/kaityo256/pycpp-parallel-pipeline/blob/main/plot.plt"
click node_figure "https://github.com/kaityo256/pycpp-parallel-pipeline/tree/main/fig"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_config,node_generator,node_job_inputs,node_task_list,node_user toneBlue
class node_entry,node_parser,node_engine,node_job_results toneAmber
class node_scheduler,node_mpi toneMint
class node_analyzer,node_summary,node_plot_script,node_gnuplot,node_figure toneRose
```

## Components

### Job preparation

- [YAML configuration](https://github.com/kaityo256/pycpp-parallel-pipeline/blob/main/input.yaml): `input.yaml`
- [Input generator](https://github.com/kaityo256/pycpp-parallel-pipeline/blob/main/generate_inputs.py): `generate_inputs.py`
- [Job input files](https://github.com/kaityo256/pycpp-parallel-pipeline/tree/main/output): `output`
- [Task list](https://github.com/kaityo256/pycpp-parallel-pipeline/blob/main/generate_inputs.py): `generate_inputs.py`

### Simulation

- [Simulation entry](https://github.com/kaityo256/pycpp-parallel-pipeline/blob/main/cpp/main.cpp): `cpp/main.cpp`
- [Parameter parser](https://github.com/kaityo256/pycpp-parallel-pipeline/tree/main/cpp/param): `cpp/param`
- [Percolation engine](https://github.com/kaityo256/pycpp-parallel-pipeline/blob/main/cpp/percolation.hpp): `cpp/percolation.hpp`
- [Per-job results](https://github.com/kaityo256/pycpp-parallel-pipeline/tree/main/output): `output`

### Parallel execution

- [CPS scheduler](https://github.com/kaityo256/pycpp-parallel-pipeline/tree/main/cps): `cps`
- **MPI runtime**

### Results and plotting

- [Result analyzer](https://github.com/kaityo256/pycpp-parallel-pipeline/blob/main/analyze_results.py): `analyze_results.py`
- [Summary data](https://github.com/kaityo256/pycpp-parallel-pipeline/tree/main/output): `output`
- [Plot definition](https://github.com/kaityo256/pycpp-parallel-pipeline/blob/main/plot.plt): `plot.plt`
- **Gnuplot**
- [Result figure](https://github.com/kaityo256/pycpp-parallel-pipeline/tree/main/fig): `fig`

### Other components

- **Workflow user**

## Connections

- Workflow user → Input generator: runs (evidence: `README.md`)
- YAML configuration → Input generator: configures (evidence: `README.md`)
- Input generator → Job input files: writes inputs (evidence: `generate_inputs.py`)
- Input generator → Task list: writes tasks (evidence: `generate_inputs.py`)
- Task list → CPS scheduler: schedules commands (evidence: `README.md`)
- MPI runtime → CPS scheduler: runs scheduler (evidence: `README.md`)
- CPS scheduler → Simulation entry: dispatches jobs (evidence: `README.md`)
- Job input files → Simulation entry: provides input (evidence: `README.md`)
- Simulation entry → Parameter parser: parses parameters (evidence: `cpp/main.cpp`)
- Simulation entry → Percolation engine: runs task (evidence: `cpp/main.cpp`)
- Percolation engine → Per-job results: writes result (evidence: `cpp/percolation.hpp`)
- Per-job results → Result analyzer: provides results (evidence: `README.md`)
- YAML configuration → Result analyzer: sets aggregation range (evidence: `analyze_results.py`)
- Result analyzer → Summary data: writes statistics (evidence: `analyze_results.py`)
- Summary data → Gnuplot: provides plot data (evidence: `README.md`)
- Plot definition → Gnuplot: configures plot (evidence: `README.md`)
- Gnuplot → Result figure: renders figure (evidence: `README.md`)