How y-ding/nurd-mlsys22-code works
Main Components
• Data Preprocessing Module: Represented by the “data” directory, which contains multiple scripts (e.g., aggregatedata.py, alibabafilter.py, timeseriesgeneration.py, etc.) responsible for processing trace data (Google and Alibaba). This module prepares input data for the machine learning experiments.
• Experimentation and Model Implementation Module: Represented by the “code” directory. Key files such as run_ts.py and run_od.py encapsulate the implementations of various prediction and outlier detection methods (e.g., gradient boosting, logistic regression, SVM with oversampling/downsampling, semi-supervised methods, Tobit regression, and Grabit). Additional helper modules (e.g., tobit.py, utils_ts.py) support the main algorithms.
• External Libraries and Dependencies: Note that the README lists dependencies (scikit-learn, imbalance-learn, pulearn, Grabit, PyOD) that are integral to the project's learning and prediction capability.
• Experimental Settings and Pipelines: The README itself reveals the project’s focus on performing experiments in Negative-Unlabeled Learning for Online Datacenter Straggler Prediction. This experimental setup, which bridges data preprocessing with distinct machine learning techniques, is a high-level architectural element.
• Data Sources: Optionally, highlight that the inputs come from real datacenter trace data, referencing Google and Alibaba traces (even though these might be fed into preprocessing scripts).
Architectural Patterns and Principles
• Modular Design: The separation between code handling experiments and data preprocessing clearly suggests a modular design, with each directory encapsulating a distinct set of responsibilities.
• Experimental Pipeline: Clearly denote the transformation flow – starting with raw trace data, moving through preprocessing (in the “data” folder), then to execution of diverse ML methods (in the “code” folder).
Final Checklist
• A Data Preprocessing block that encapsulates all pre-processing scripts in the “data” folder.
• A section for External Libraries to show dependency relationships.
• Clearly label and annotate each component with its role in the overall architecture.