# b1n-isme/thesis_2 architecture

> Architecture diagram of the GitHub repository b1n-isme/thesis_2, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-06-01). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/b1n-isme/thesis_2
- Repository: https://github.com/b1n-isme/thesis_2

## Overview

##### Project Type and Purpose
- This is a batch-oriented, full-stack data pipeline for Bitcoin price forecasting.
- It ingests raw time-series data, preprocesses it, runs optional hyperparameter optimization (HPO) across three modeling frameworks (statsforecast, mlforecast, neuralforecast), compares models, and emits final forecasts and reports.

##### File Structure
- Top-level entry points:
- main.py / pipeline.py orchestrate end-to-end workflow
- test.ipynb / test.py validate functionality
- config/: centralized configuration (config/base.py defines horizons, CV folds, HPO samples)
- data/: raw and final datasets (CSV / Parquet)
- src/: core modules subdivided into
- dataset/ (data loading & preprocessing)
- models/ (model_registry, per-framework model definitions)
- pipelines/ (data preparation, model evaluation, forecasting, results processing, visualization)
- utils/ (generic helpers)
- results/: HPO results, CV metrics, final summaries, plots, saved models

##### Extract Architectural Insights from README
- Single entry point (`main_clean.py`) invoking four sequential steps:
1. Data Preparation
2. Hyperparameter Optimization (optional)
3. Model Comparison
4. Final Evaluation & Reporting
- ModelRegistry implements a plugin pattern: unified interface to instantiate auto-models (for HPO) and standard models.
- Clear separation: configuration → data → models → pipelines → outputs.

##### Main Components & Their Interactions
a. CLI / Orchestrator
- main_clean.py reads command-line flags, loads config, dispatches pipeline steps.
b. Configuration Module
- config/base.py supplies global constants (horizon, CV folds, sample counts).
c. Data Ingestion & Preparation
- src/dataset/data_preparation.py loads Parquet/CSV, engineers exogenous features, splits train/test.
d. Model Registry & HPO Engine
- src/models/model_registry.py (and auto_models.py) registers model classes.
- HPO scripts (pipeline or src/pipelines/pipeline_setup.py) invoke auto-models with cross-validation.
e. Model Training & Evaluation Pipelines
- src/pipelines/model_evaluation.py runs CV, computes MAE/RMSE.
- src/pipelines/model_forecasting.py generates forecasts on hold-out.
- src/pipelines/results_processing.py ranks and summarizes.
f. Visualization Layer
- src/pipelines/visualization.py / src/utils/utils.py for plotting and report formatting.
g. Output Sink
- results/ directory: organized into hpo/, cv/, final/, plot/ subfolders.

##### Architectural Patterns & Design Principles
- Layered, modular design: each folder is a logical layer
- Plugin pattern for adding new models (ModelRegistry)
- Single entry point simplifies orchestration
- Config-driven behavior (fast-mode, skip-hpo)
- Clear separation of concerns (data vs. models vs. evaluation vs. visualization)

## Diagram (Mermaid)

```mermaid
flowchart TD
    %% Step 1: Orchestrator & Config
    subgraph "0 Orchestration & Configuration"
        direction TB
        CLI["CLI / Orchestrator"]:::config
        Config["Configuration Module"]:::config
    end

    %% Step 1: Data Preparation
    subgraph "1 Data Preparation" 
        direction TB
        RawData["Raw Time-Series Data (CSV/Parquet)"]:::data
        DataPrepModule["Data Preparation Module"]:::processing
        DatasetPrep["Dataset Prep Functions"]:::processing
    end

    %% Step 2: Hyperparameter Optimization
    subgraph "2 Hyperparameter Optimization (HPO)"
        direction TB
        HPOEngine["HPO Engine"]:::processing
        ModelRegistry["Model Registry (Plugin)"]:::modeling
        AutoMLF["MLForecast Auto-Models"]:::modeling
        AutoNF["NeuralForecast HPO Config"]:::modeling
    end

    %% Step 3: Model Comparison & Evaluation
    subgraph "3 Model Comparison & Evaluation"
        direction TB
        StatsModels["StatsForecast Models"]:::modeling
        MLModels["MLForecast Models"]:::modeling
        NNModels["NeuralForecast Models"]:::modeling
        EvalCV["Model Evaluation (CV & Metrics)"]:::processing
        Forecast["Hold-out Forecasting"]:::processing
        ResultsProc["Results Processing"]:::processing
    end

    %% Step 4: Visualization & Reporting
    subgraph "4 Visualization & Reporting"
        direction TB
        Visualization["Visualization & Reporting"]:::processing
        Utils["Utility Helpers"]:::processing
    end

    %% Output Storage
    Results["Results Storage"]:::storage

    %% Flow
    CLI -->|"reads flags"| Config
    CLI -->|"invokes pipeline"| RawData
    Config -->|provides params| DataPrepModule
    RawData --> DataPrepModule
    DataPrepModule --> DatasetPrep
    DatasetPrep --> HPOEngine

    HPOEngine --> ModelRegistry
    ModelRegistry --> AutoMLF
    ModelRegistry --> AutoNF

    subgraph "Framework Models"
        direction TB
        StatsModels
        MLModels
        NNModels
    end

    AutoMLF --> MLModels
    AutoNF --> NNModels
    StatsModels --> EvalCV
    MLModels --> EvalCV
    NNModels --> EvalCV

    EvalCV --> Forecast
    Forecast --> ResultsProc
    ResultsProc --> Visualization
    Visualization --> Results

    Utils --> Visualization

    %% Click Events
    click CLI "https://github.com/b1n-isme/thesis_2/blob/master/main.py"
    click CLI "https://github.com/b1n-isme/thesis_2/blob/master/pipeline.py"
    click Config "https://github.com/b1n-isme/thesis_2/blob/master/config/base.py"
    click Config "https://github.com/b1n-isme/thesis_2/blob/master/config/search_algo.py"
    click Config "https://github.com/b1n-isme/thesis_2/blob/master/config/algo.txt"
    click RawData "https://github.com/b1n-isme/thesis_2/tree/master/data/final/"
    click DataPrepModule "https://github.com/b1n-isme/thesis_2/blob/master/src/dataset/data_preparation.py"
    click DatasetPrep "https://github.com/b1n-isme/thesis_2/blob/master/src/pipelines/data_preparation.py"
    click HPOEngine "https://github.com/b1n-isme/thesis_2/blob/master/src/pipelines/pipeline_setup.py"
    click AutoMLF "https://github.com/b1n-isme/thesis_2/blob/master/src/models/mlforecast/auto_models.py"
    click AutoNF "https://github.com/b1n-isme/thesis_2/blob/master/src/models/neuralforecast/auto_cfg.py"
    click StatsModels "https://github.com/b1n-isme/thesis_2/blob/master/src/models/statsforecast/models.py"
    click MLModels "https://github.com/b1n-isme/thesis_2/blob/master/src/models/mlforecast/models.py"
    click NNModels "https://github.com/b1n-isme/thesis_2/blob/master/src/models/neuralforecast/models.py"
    click EvalCV "https://github.com/b1n-isme/thesis_2/blob/master/src/pipelines/model_evaluation.py"
    click Forecast "https://github.com/b1n-isme/thesis_2/blob/master/src/pipelines/model_forecasting.py"
    click ResultsProc "https://github.com/b1n-isme/thesis_2/blob/master/src/pipelines/results_processing.py"
    click Visualization "https://github.com/b1n-isme/thesis_2/blob/master/src/pipelines/visualization.py"
    click Utils "https://github.com/b1n-isme/thesis_2/blob/master/src/utils/utils.py"
    click Results "https://github.com/b1n-isme/thesis_2/tree/master/results/"
    click Results "https://github.com/b1n-isme/thesis_2/tree/master/results/final/"
    click Results "https://github.com/b1n-isme/thesis_2/tree/master/results/plot/"

    %% Styles
    classDef data fill:#ccffcc,stroke:#333,stroke-width:1px
    classDef processing fill:#cce5ff,stroke:#333,stroke-width:1px
    classDef modeling fill:#ffdd99,stroke:#333,stroke-width:1px
    classDef config fill:#e2ccff,stroke:#333,stroke-width:1px
    classDef storage fill:#e0e0e0,stroke:#333,stroke-width:1px
```
