How hayato-n8810/hayalab works
Project Type & Purpose
• This is neither a classical web app nor a microservice product; it’s a research‐oriented data‐processing pipeline written in Python (with some JS/Notebook support).
High-Level File Structure
• Top level:
• Docker/Docker-Compose files – containerize various steps.
• src/ – core pipeline modules, subdivided by function (collect_repo, trace, abst, pattern, measure, grouping, MBquery_analysis, util, const).
• results/ – outputs of each stage.
• archive/ – historical snapshots of past research topics (Omori, Zenkoku).
• README documents the purpose and module layout.
Key Components & Responsibilities
• Data Ingestion (src/collect_repo): clones JS repos from GitHub, outputs raw repos.
• Trace (src/trace): walks commit history and diffs to extract code-change snippets.
• Abstraction (src/abst): transforms raw JS into abstract representations.
• Pattern Detection (src/pattern): identifies known slow code patterns from diffs.
• Microbenchmarking (src/measure): runs benchmarks on pattern variants to measure speed differences.
• Grouping & Analysis (src/grouping, src/MBquery_analysis): clusters benchmark results, matches patterns, prepares plots.
• Utilities & Constants (src/util, src/const): shared helper scripts and configuration constants.
• Results Storage (results/, plus each module’s sub-folders): CSV, JSON, plot images.
External Dependencies & Infrastructure
• Docker containers for isolation of clone, measure, and other stages (Dockerfiles under src/collect_repo and src/measure).
• Docker-Compose orchestrates multi-container workflows.
• Jupyter notebooks (in archive) for ad-hoc visualization.
• UV runner (“uv run python …”) for uniform module execution.
• NPM for jsparser.js.
Architectural Patterns & Design Principles
• Pipeline/ETL pattern: clearly staged transformations.
• Modular layering: each “src/<component>” is a self-contained service.
• Reproducibility via containers: Dockerfiles + docker-compose.
• Separation of concerns: data collection, trace, abstraction, detection, benchmarking, grouping.
Technologies & Frameworks to Call Out
• Python (main language), Jupyter notebooks (visualization), JavaScript parser (jsparser.js)
• Docker, Docker-Compose for environment control
• JSON/CSV for interchange formats
• UV (CLI tool) for running scripts
Detail Level
• Essential interactions—especially the reuse of util/const across modules.