How sstanoje/reproducible-short-running-benchmarking works
This repository is a reproducibility artifact for short-running benchmark experiments. Its visible runtime workflow is to analyze collected benchmark measurements: suite-specific processors calculate timing statistics, stability scripts compare RMAD across iteration counts, and separate scripts produce warmup evidence and Wilcoxon reports. The README also describes system setup and benchmark execution, but those source directories are absent from the supplied tree, so they are not mapped here. Analysis-script relationships to measurements are documented; connections between generated outputs are omitted where source evidence is insufficient.
Main entry points: results Experimental Results · dacapo/process_file.py DaCapo Statistics · stability_per_iterations.py DaCapo Stability · dacapo_con_scala/process_file.py Scala DaCapo Statistics
Components 17
Measurements
- Experimental Results
results
Analysis
- DaCapo Statistics
analysis_scripts/process_results_scripts/dacapo/process_file.py - DaCapo Stability
analysis_scripts/process_results_scripts/dacapo/stability_per_iterations.py - Scala DaCapo Statistics
analysis_scripts/process_results_scripts/dacapo_con_scala/process_file.py - Scala DaCapo Stability
analysis_scripts/process_results_scripts/dacapo_con_scala/stability_per_iterations.py - PARSEC Statistics
analysis_scripts/process_results_scripts/parsec/process_file.py - PARSEC Stability
analysis_scripts/process_results_scripts/parsec/stability_per_iterations.py - PolyBench Statistics
analysis_scripts/process_results_scripts/polyBenchC/process_file.py - PolyBench Stability
analysis_scripts/process_results_scripts/polyBenchC/stability_per_iterations.py - Renaissance Statistics
analysis_scripts/process_results_scripts/renaissance/process_file.py - Renaissance Stability
analysis_scripts/process_results_scripts/renaissance/stability_per_iterations.py - Warmup Analysis
analysis_scripts/warmup_scripts/calculate_warmup.py - Wilcoxon Analysis
analysis_scripts/statistics_scripts/calculate_wilcoxon.py
Reports
- Stability Outputs
stability_per_iterations_results - Warmup Output
warmup_analysis_results - Wilcoxon Outputs
wilcoxon_test_results
Other components
- Researcher
Connections 27
- Researcher → DaCapo Statistics: runs
- Researcher → DaCapo Stability: runs
- Researcher → Scala DaCapo Statistics: runs
- Researcher → Scala DaCapo Stability: runs
- Researcher → PARSEC Statistics: runs
- Researcher → PARSEC Stability: runs
- Researcher → PolyBench Statistics: runs
- Researcher → PolyBench Stability: runs
- Researcher → Renaissance Statistics: runs
- Researcher → Renaissance Stability: runs
- Researcher → Warmup Analysis: runs
- Researcher → Wilcoxon Analysis: runs
- Experimental Results → DaCapo Statistics: provides measurements
README.md - Experimental Results → DaCapo Stability: provides measurements
README.md - Experimental Results → Scala DaCapo Statistics: provides measurements
README.md - Experimental Results → Scala DaCapo Stability: provides measurements
README.md - Experimental Results → PARSEC Statistics: provides measurements
README.md - Experimental Results → PARSEC Stability: provides measurements
README.md - Experimental Results → PolyBench Statistics: provides measurements
README.md - Experimental Results → PolyBench Stability: provides measurements
README.md - Experimental Results → Renaissance Statistics: provides measurements
README.md - Experimental Results → Renaissance Stability: provides measurements
README.md - Experimental Results → Warmup Analysis: reads measurements
analysis_scripts/warmup_scripts/calculate_warmup.py - Experimental Results → Wilcoxon Analysis: reads summaries
analysis_scripts/statistics_scripts/calculate_wilcoxon.py - Researcher → Warmup Output: generates
README.md - Researcher → Wilcoxon Outputs: generates
README.md - Researcher → Stability Outputs: generates
README.md