How kanaji2002/ci-skip works
Project type and purpose
• This is a data-replication / data-curation pipeline written in Python.
• Its goal is to fetch raw data (commit histories, CI data, file metadata) from GitHub (and possibly CI services), transform and merge these pieces, and produce final datasets and analysis artifacts for a research paper’s replication appendix.
File structure
• Top-level:
• data/replication – all replication materials and pipelines
• README.md – (placeholder)
• Under data/replication/data-curation:
• “pipeline” scripts: 1_1, 1_2, 2_… – staged ETL steps
• clone_and_get_data.py, get_ci_data.py, get_dependencies_for_each_commit.py, get_version_changing_commits.py – discrete data-fetch modules
• data_downloader/script – wrappers around external APIs (GitHub, giPython)
• create_dataset.py – consolidates processed inputs into final dataset
• Other files: zipped archives (datasets.zip, data-analysis.zip), Online Appendix.pdf
• No server or UI code – purely batch pipelines and data artifacts.
README for additional insights
• (README not yet populated)
• Rely on file names to infer that each numbered pipeline step builds on the previous.
Essential details to capture
• All numbered pipeline steps in order
• The two wrapper layers (common vs. specific) under data_downloader
• The transformation from raw fetch to merged dataset
• Final outputs (datasets.zip, data-analysis.zip, Online Appendix.pdf)