How koraykarabina/impacx_onhw works
Main System Components
• Data Layer:
– Dataset folder: holds the raw OnHW-chars dataset (stored as NumPy arrays and associated readme files).
• Preprocessing / Feature Extraction:
– Scripts such as OnHW_ML_extract_features_various_nsig.py handle feature extraction (using tools like tsfresh).
– Utility modules in the utils/ folder (e.g., config.py) provide configuration for preprocessing.
• Machine Learning Pipeline:
– Baseline ML training scripts like OnHW_ML_DL_baseline_and_optimized_train_and_eval.py that call functions for data filtering, feature extraction, and standard model training (e.g., KNN, logistic regression, SVM, etc.).
– Training scripts for specialized ML tasks: OnHW_ML_train_KNN_various_nsig.py and OnHW_ML_train_KNN_various_ncomp.py for training KNN models over different feature/n_component settings.
• Deep Learning Pipeline:
– Deep learning scripts (e.g., OnHW_DL_* scripts) to train standard deep learning models.
– Optimization module: OnHW_DL_optimize.py uses the Optuna framework to optimize deep learning model parameters.
• Evaluation & Ensemble:
– Scripts such as ensemble_evaluation.py process evaluation metrics, generate classification reports, confusion matrices, and perform ensemble voting (majority, soft voting, weighted, etc.).
• Visualization & Reporting:
– Results plotting: plot_kNN_results.py to produce figures (e.g., Figures 3 and 4) using matplotlib.
– Results aggregation and visualization are handled by results_to_visual.py, which creates HTML reports.
• Explainability & Failure Analysis:
– Explain.py produces per-fold explainability outputs (bar charts and detailed time series graphs).
– Failure analysis modules: failure_space_analysis.py and failure_space_array.py provide insights into where models err (bar graphs and matrices).
• Utility Modules:
– The “utils” folder includes config, helpers, and routines for file/folder management which are used across the entire pipeline for consistency.
Interactions and Data Flow
– Data begins at the dataset layer: the raw dataset is ingested.
– Preprocessing Stage: Data is pre-processed, features are extracted (with configurations driven by utils/config.py).
– Training Stage: There is separate branching:
• ML Training: Baseline and metric learned versions run sequentially using the preprocessed features.
• DL Training: Baseline deep learning models and optimized deep learning models are trained and evaluated.
– Evaluation Stage: After training, results are stored in a designated “output” directory (and sub-folders such as ml_results, dl_results, etc.). Various scripts perform:
• Model evaluation (including ensemble voting).
• Failure space and explainability analyses.
– Visualization Stage: Final results, accuracies, confusion matrices, and performance plots are generated and aggregated to produce visual reports.
Architectural Patterns and Technical Aspects
– Pipeline/Workflow Architecture: The project operates as a sequential series of steps (data → feature extraction → training → evaluation → visualization), with several branching points for ML vs. DL.
– Modularity: The code is separated into multiple single-purpose scripts to allow easier re-running and tracking of experiments.
– Configurability: The configuration in utils/config.py and use of external dependencies (e.g., tsfresh, optuna) shows design for experiment reproducibility.
– Data Persistence: Intermediate and final outputs are stored on disk in a structured “output” directory; this is key for later analysis and visualization.