# sstanoje/reproducible-short-running-benchmarking architecture

> Architecture diagram of the GitHub repository sstanoje/reproducible-short-running-benchmarking, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-10-07). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/sstanoje/reproducible-short-running-benchmarking
- Repository: https://github.com/sstanoje/reproducible-short-running-benchmarking

## Overview

This repository is a reproducibility artifact for short-running benchmark experiments. Its visible runtime workflow is to analyze collected benchmark measurements: suite-specific processors calculate timing statistics, stability scripts compare RMAD across iteration counts, and separate scripts produce warmup evidence and Wilcoxon reports. The README also describes system setup and benchmark execution, but those source directories are absent from the supplied tree, so they are not mapped here. Analysis-script relationships to measurements are documented; connections between generated outputs are omitted where source evidence is insufficient.

Main entry points: [`results`](https://github.com/sstanoje/reproducible-short-running-benchmarking/tree/main/results) (Experimental Results), [`analysis_scripts/process_results_scripts/dacapo/process_file.py`](https://github.com/sstanoje/reproducible-short-running-benchmarking/blob/main/analysis_scripts/process_results_scripts/dacapo/process_file.py) (DaCapo Statistics), [`analysis_scripts/process_results_scripts/dacapo/stability_per_iterations.py`](https://github.com/sstanoje/reproducible-short-running-benchmarking/blob/main/analysis_scripts/process_results_scripts/dacapo/stability_per_iterations.py) (DaCapo Stability), [`analysis_scripts/process_results_scripts/dacapo_con_scala/process_file.py`](https://github.com/sstanoje/reproducible-short-running-benchmarking/blob/main/analysis_scripts/process_results_scripts/dacapo_con_scala/process_file.py) (Scala DaCapo Statistics)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_measurements["Measurements"]
  node_raw_results[("Experimental Results")]
end

subgraph group_processing["Analysis"]
  node_dacapo_process["DaCapo Statistics<br/>[process_file.py]"]
  node_dacapo_stability["DaCapo Stability"]
  node_scala_process["Scala DaCapo Statistics<br/>[process_file.py]"]
  node_scala_stability["Scala DaCapo Stability"]
  node_parsec_process["PARSEC Statistics<br/>[process_file.py]"]
  node_parsec_stability["PARSEC Stability"]
  node_polybench_process["PolyBench Statistics<br/>[process_file.py]"]
  node_polybench_stability["PolyBench Stability"]
  node_renaissance_process["Renaissance Statistics<br/>[process_file.py]"]
  node_renaissance_stability["Renaissance Stability"]
  node_warmup["Warmup Analysis"]
  node_wilcoxon["Wilcoxon Analysis"]
end

subgraph group_reports["Reports"]
  node_stability_reports["Stability Outputs"]
  node_warmup_reports["Warmup Output"]
  node_wilcoxon_reports["Wilcoxon Outputs"]
end

node_analyst(("Researcher"))

node_analyst -->|"runs"| node_dacapo_process
node_analyst -->|"runs"| node_dacapo_stability
node_analyst -->|"runs"| node_scala_process
node_analyst -->|"runs"| node_scala_stability
node_analyst -->|"runs"| node_parsec_process
node_analyst -->|"runs"| node_parsec_stability
node_analyst -->|"runs"| node_polybench_process
node_analyst -->|"runs"| node_polybench_stability
node_analyst -->|"runs"| node_renaissance_process
node_analyst -->|"runs"| node_renaissance_stability
node_analyst -->|"runs"| node_warmup
node_analyst -->|"runs"| node_wilcoxon
node_raw_results -.->|"provides measurements"| node_dacapo_process
node_raw_results -.->|"provides measurements"| node_dacapo_stability
node_raw_results -.->|"provides measurements"| node_scala_process
node_raw_results -.->|"provides measurements"| node_scala_stability
node_raw_results -.->|"provides measurements"| node_parsec_process
node_raw_results -.->|"provides measurements"| node_parsec_stability
node_raw_results -.->|"provides measurements"| node_polybench_process
node_raw_results -.->|"provides measurements"| node_polybench_stability
node_raw_results -.->|"provides measurements"| node_renaissance_process
node_raw_results -.->|"provides measurements"| node_renaissance_stability
node_raw_results -->|"reads measurements"| node_warmup
node_raw_results -->|"reads summaries"| node_wilcoxon
node_analyst -.->|"generates"| node_warmup_reports
node_analyst -.->|"generates"| node_wilcoxon_reports
node_analyst -.->|"generates"| node_stability_reports

click node_raw_results "https://github.com/sstanoje/reproducible-short-running-benchmarking/tree/main/results"
click node_dacapo_process "https://github.com/sstanoje/reproducible-short-running-benchmarking/blob/main/analysis_scripts/process_results_scripts/dacapo/process_file.py"
click node_dacapo_stability "https://github.com/sstanoje/reproducible-short-running-benchmarking/blob/main/analysis_scripts/process_results_scripts/dacapo/stability_per_iterations.py"
click node_scala_process "https://github.com/sstanoje/reproducible-short-running-benchmarking/blob/main/analysis_scripts/process_results_scripts/dacapo_con_scala/process_file.py"
click node_scala_stability "https://github.com/sstanoje/reproducible-short-running-benchmarking/blob/main/analysis_scripts/process_results_scripts/dacapo_con_scala/stability_per_iterations.py"
click node_parsec_process "https://github.com/sstanoje/reproducible-short-running-benchmarking/blob/main/analysis_scripts/process_results_scripts/parsec/process_file.py"
click node_parsec_stability "https://github.com/sstanoje/reproducible-short-running-benchmarking/blob/main/analysis_scripts/process_results_scripts/parsec/stability_per_iterations.py"
click node_polybench_process "https://github.com/sstanoje/reproducible-short-running-benchmarking/blob/main/analysis_scripts/process_results_scripts/polyBenchC/process_file.py"
click node_polybench_stability "https://github.com/sstanoje/reproducible-short-running-benchmarking/blob/main/analysis_scripts/process_results_scripts/polyBenchC/stability_per_iterations.py"
click node_renaissance_process "https://github.com/sstanoje/reproducible-short-running-benchmarking/blob/main/analysis_scripts/process_results_scripts/renaissance/process_file.py"
click node_renaissance_stability "https://github.com/sstanoje/reproducible-short-running-benchmarking/blob/main/analysis_scripts/process_results_scripts/renaissance/stability_per_iterations.py"
click node_warmup "https://github.com/sstanoje/reproducible-short-running-benchmarking/blob/main/analysis_scripts/warmup_scripts/calculate_warmup.py"
click node_wilcoxon "https://github.com/sstanoje/reproducible-short-running-benchmarking/blob/main/analysis_scripts/statistics_scripts/calculate_wilcoxon.py"
click node_stability_reports "https://github.com/sstanoje/reproducible-short-running-benchmarking/tree/main/stability_per_iterations_results"
click node_warmup_reports "https://github.com/sstanoje/reproducible-short-running-benchmarking/tree/main/warmup_analysis_results"
click node_wilcoxon_reports "https://github.com/sstanoje/reproducible-short-running-benchmarking/tree/main/wilcoxon_test_results"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_raw_results toneBlue
class node_dacapo_process,node_dacapo_stability,node_scala_process,node_scala_stability,node_parsec_process,node_parsec_stability,node_polybench_process,node_polybench_stability,node_renaissance_process,node_renaissance_stability,node_warmup,node_wilcoxon toneAmber
class node_stability_reports,node_warmup_reports,node_wilcoxon_reports toneMint
class node_analyst toneIndigo
```

## Components

### Measurements

- [Experimental Results](https://github.com/sstanoje/reproducible-short-running-benchmarking/tree/main/results): `results`

### Analysis

- [DaCapo Statistics](https://github.com/sstanoje/reproducible-short-running-benchmarking/blob/main/analysis_scripts/process_results_scripts/dacapo/process_file.py): `analysis_scripts/process_results_scripts/dacapo/process_file.py`
- [DaCapo Stability](https://github.com/sstanoje/reproducible-short-running-benchmarking/blob/main/analysis_scripts/process_results_scripts/dacapo/stability_per_iterations.py): `analysis_scripts/process_results_scripts/dacapo/stability_per_iterations.py`
- [Scala DaCapo Statistics](https://github.com/sstanoje/reproducible-short-running-benchmarking/blob/main/analysis_scripts/process_results_scripts/dacapo_con_scala/process_file.py): `analysis_scripts/process_results_scripts/dacapo_con_scala/process_file.py`
- [Scala DaCapo Stability](https://github.com/sstanoje/reproducible-short-running-benchmarking/blob/main/analysis_scripts/process_results_scripts/dacapo_con_scala/stability_per_iterations.py): `analysis_scripts/process_results_scripts/dacapo_con_scala/stability_per_iterations.py`
- [PARSEC Statistics](https://github.com/sstanoje/reproducible-short-running-benchmarking/blob/main/analysis_scripts/process_results_scripts/parsec/process_file.py): `analysis_scripts/process_results_scripts/parsec/process_file.py`
- [PARSEC Stability](https://github.com/sstanoje/reproducible-short-running-benchmarking/blob/main/analysis_scripts/process_results_scripts/parsec/stability_per_iterations.py): `analysis_scripts/process_results_scripts/parsec/stability_per_iterations.py`
- [PolyBench Statistics](https://github.com/sstanoje/reproducible-short-running-benchmarking/blob/main/analysis_scripts/process_results_scripts/polyBenchC/process_file.py): `analysis_scripts/process_results_scripts/polyBenchC/process_file.py`
- [PolyBench Stability](https://github.com/sstanoje/reproducible-short-running-benchmarking/blob/main/analysis_scripts/process_results_scripts/polyBenchC/stability_per_iterations.py): `analysis_scripts/process_results_scripts/polyBenchC/stability_per_iterations.py`
- [Renaissance Statistics](https://github.com/sstanoje/reproducible-short-running-benchmarking/blob/main/analysis_scripts/process_results_scripts/renaissance/process_file.py): `analysis_scripts/process_results_scripts/renaissance/process_file.py`
- [Renaissance Stability](https://github.com/sstanoje/reproducible-short-running-benchmarking/blob/main/analysis_scripts/process_results_scripts/renaissance/stability_per_iterations.py): `analysis_scripts/process_results_scripts/renaissance/stability_per_iterations.py`
- [Warmup Analysis](https://github.com/sstanoje/reproducible-short-running-benchmarking/blob/main/analysis_scripts/warmup_scripts/calculate_warmup.py): `analysis_scripts/warmup_scripts/calculate_warmup.py`
- [Wilcoxon Analysis](https://github.com/sstanoje/reproducible-short-running-benchmarking/blob/main/analysis_scripts/statistics_scripts/calculate_wilcoxon.py): `analysis_scripts/statistics_scripts/calculate_wilcoxon.py`

### Reports

- [Stability Outputs](https://github.com/sstanoje/reproducible-short-running-benchmarking/tree/main/stability_per_iterations_results): `stability_per_iterations_results`
- [Warmup Output](https://github.com/sstanoje/reproducible-short-running-benchmarking/tree/main/warmup_analysis_results): `warmup_analysis_results`
- [Wilcoxon Outputs](https://github.com/sstanoje/reproducible-short-running-benchmarking/tree/main/wilcoxon_test_results): `wilcoxon_test_results`

### Other components

- **Researcher**

## Connections

- Researcher → DaCapo Statistics: runs
- Researcher → DaCapo Stability: runs
- Researcher → Scala DaCapo Statistics: runs
- Researcher → Scala DaCapo Stability: runs
- Researcher → PARSEC Statistics: runs
- Researcher → PARSEC Stability: runs
- Researcher → PolyBench Statistics: runs
- Researcher → PolyBench Stability: runs
- Researcher → Renaissance Statistics: runs
- Researcher → Renaissance Stability: runs
- Researcher → Warmup Analysis: runs
- Researcher → Wilcoxon Analysis: runs
- Experimental Results → DaCapo Statistics: provides measurements (evidence: `README.md`)
- Experimental Results → DaCapo Stability: provides measurements (evidence: `README.md`)
- Experimental Results → Scala DaCapo Statistics: provides measurements (evidence: `README.md`)
- Experimental Results → Scala DaCapo Stability: provides measurements (evidence: `README.md`)
- Experimental Results → PARSEC Statistics: provides measurements (evidence: `README.md`)
- Experimental Results → PARSEC Stability: provides measurements (evidence: `README.md`)
- Experimental Results → PolyBench Statistics: provides measurements (evidence: `README.md`)
- Experimental Results → PolyBench Stability: provides measurements (evidence: `README.md`)
- Experimental Results → Renaissance Statistics: provides measurements (evidence: `README.md`)
- Experimental Results → Renaissance Stability: provides measurements (evidence: `README.md`)
- Experimental Results → Warmup Analysis: reads measurements (evidence: `analysis_scripts/warmup_scripts/calculate_warmup.py`)
- Experimental Results → Wilcoxon Analysis: reads summaries (evidence: `analysis_scripts/statistics_scripts/calculate_wilcoxon.py`)
- Researcher → Warmup Output: generates (evidence: `README.md`)
- Researcher → Wilcoxon Outputs: generates (evidence: `README.md`)
- Researcher → Stability Outputs: generates (evidence: `README.md`)
