How b1n-isme/thesis_2 works
Project Type and Purpose
• This is a batch-oriented, full-stack data pipeline for Bitcoin price forecasting.
• It ingests raw time-series data, preprocesses it, runs optional hyperparameter optimization (HPO) across three modeling frameworks (statsforecast, mlforecast, neuralforecast), compares models, and emits final forecasts and reports.
File Structure
• Top-level entry points:
• main.py / pipeline.py orchestrate end-to-end workflow
• test.ipynb / test.py validate functionality
• config/: centralized configuration (config/base.py defines horizons, CV folds, HPO samples)
• data/: raw and final datasets (CSV / Parquet)
• src/: core modules subdivided into
• dataset/ (data loading & preprocessing)
• models/ (model_registry, per-framework model definitions)
• pipelines/ (data preparation, model evaluation, forecasting, results processing, visualization)
• utils/ (generic helpers)
• results/: HPO results, CV metrics, final summaries, plots, saved models
Extract Architectural Insights from README
• Single entry point (main_clean.py) invoking four sequential steps:
1. Data Preparation
2. Hyperparameter Optimization (optional)
3. Model Comparison
4. Final Evaluation & Reporting
• ModelRegistry implements a plugin pattern: unified interface to instantiate auto-models (for HPO) and standard models.
• Clear separation: configuration → data → models → pipelines → outputs.
Main Components & Their Interactions
a. CLI / Orchestrator
• main_clean.py reads command-line flags, loads config, dispatches pipeline steps.
b. Configuration Module
• config/base.py supplies global constants (horizon, CV folds, sample counts).
c. Data Ingestion & Preparation
• src/dataset/data_preparation.py loads Parquet/CSV, engineers exogenous features, splits train/test.
d. Model Registry & HPO Engine
• src/models/model_registry.py (and auto_models.py) registers model classes.
• HPO scripts (pipeline or src/pipelines/pipeline_setup.py) invoke auto-models with cross-validation.
e. Model Training & Evaluation Pipelines
• src/pipelines/model_evaluation.py runs CV, computes MAE/RMSE.
• src/pipelines/model_forecasting.py generates forecasts on hold-out.
• src/pipelines/results_processing.py ranks and summarizes.
f. Visualization Layer
• src/pipelines/visualization.py / src/utils/utils.py for plotting and report formatting.
g. Output Sink
• results/ directory: organized into hpo/, cv/, final/, plot/ subfolders.
Architectural Patterns & Design Principles
• Layered, modular design: each folder is a logical layer
• Plugin pattern for adding new models (ModelRegistry)
• Single entry point simplifies orchestration
• Config-driven behavior (fast-mode, skip-hpo)
• Clear separation of concerns (data vs. models vs. evaluation vs. visualization)