# lldelduca/btc_predictions_denoisingautoencoders architecture

> Architecture diagram of the GitHub repository lldelduca/btc_predictions_denoisingautoencoders, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-06-06). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/lldelduca/btc_predictions_denoisingautoencoders
- Repository: https://github.com/lldelduca/btc_predictions_denoisingautoencoders

## Overview

##### Project Type and Purpose
- Type: Data-science/machine-learning pipeline contained in a Jupyter notebook.
- Purpose:
– Train and compare denoising autoencoders (DAE) versus standard autoencoders on MNIST for feature extraction and noise reduction.
– Use those extracted features (plus raw financial indicators) to build and evaluate Bitcoin price predictors (linear regression, SVR, LSTM).
– Benchmark against a simple moving average and run statistical tests on residuals.

##### File Structure
- Only file: Full_Notebook.ipynb
– Sections inside notebook correspond to distinct functional stages: Data Acquisition, Preprocessing, Model Definitions, Training/Validation, Prediction, Evaluation, Visualization.

##### README Insights
- Data sources: MNIST dataset, custom web scrapers, TA-Lib, Yahoo Finance YQL API.
- Models: Stacked Autoencoder (SAE), Stacked Denoising Autoencoder (SDAE), Linear Regression, SVR, LSTM.
- Hyperparameter tuning: Optuna.
- Evaluation metrics: MAE, RMSE, MAPE; statistical tests (Durbin-Watson, White’s, Jarque-Bera).
- Visualization: plots of reconstructions, residuals, kernel densities, prediction vs. benchmark.

##### Main Components and Their Interactions
a. Data Acquisition
– MNIST loader (PyTorch)
– Financial data scrapers (TA-Lib, Yahoo API)
b. Data Preprocessing & Feature Engineering
– Train/validation/test splits
– Noise injection (masking) for DAE
– Feature scaling, lagged price creation
c. Encoding Stage
– SAE training (encoder/decoder)
– SDAE training (with noise)
– Output: reduced-dimension feature vectors
d. Feature Selection & Combination
– Compare SAE, SDAE, and raw feature sets
– Select robust features based on reconstruction performance
e. Predictive Modeling
– Linear Regression (with statistical validation)
– Support Vector Regression
– LSTM network
– Hyperparameter tuning via Optuna
f. Benchmarking & Evaluation
– Simple Moving Average baseline
– Compute MAE, RMSE, MAPE
– Conduct Durbin-Watson, White’s, Jarque-Bera tests
g. Visualization & Reporting
– Reconstruction examples (original/noisy/reconstructed)
– Prediction vs. actual price plots
– Residual and kernel density plots

##### Architectural Patterns & Design Principles
- Pipeline pattern: clear separation between data ingestion, preprocessing, feature extraction, modeling, evaluation, and visualization.
- Modularity inside notebook: each section implements one stage.
- Reuse of encoders for two purposes (feature reduction + denoising).
- Experimentation loop: swap feature sets and model types.

##### Technologies & Frameworks
– Jupyter Notebook – orchestration layer
– PyTorch – SAE & SDAE
– Scikit-Learn – regression, SVR, preprocessing
– Keras or PyTorch – LSTM (depending on implementation)
– Optuna – hyperparameter tuning
– TA-Lib, yfinance – feature scraping
– pandas, NumPy, matplotlib/seaborn – data manipulation & plotting

## Diagram (Mermaid)

```mermaid
flowchart TD
    %% Data Sources
    subgraph "Data Acquisition"
        MNIST["MNIST Data Loader"]:::data
        Finance["Financial Data Scrapers"]:::data
    end

    %% Preprocessing
    Preproc["Splitting, Scaling, Noise Injection, Lag Features"]:::process

    %% Encoding
    subgraph "Encoding Stage"
        SAE["SAE Training"]:::model
        SDAE["SDAE Training"]:::model
    end

    %% Feature Selection
    FS["Feature Selection & Combination"]:::process

    %% Predictive Modeling
    subgraph "Predictive Modeling"
        LR["Linear Regression + Optuna"]:::model
        SVR["SVR + Optuna"]:::model
        LSTM["LSTM + Optuna"]:::model
    end

    %% Evaluation and Visualization
    Eval{"Evaluation & Statistical Tests"}:::decision
    Viz["Visualization & Reporting"]:::process

    %% Documentation
    subgraph "Project Documentation"
        README["README.md"]:::data
        LICENSE["LICENSE"]:::data
    end

    %% Data Flow
    MNIST -->|"raw images"| Preproc
    Finance -->|"financial indicators"| Preproc

    Preproc -->|"noisy images"| SAE
    Preproc -->|"noisy images"| SDAE
    Preproc -->|"raw features"| FS
    SAE -->|"feature vectors"| FS
    SDAE -->|"feature vectors"| FS

    FS -->|"features"| LR
    FS -->|"features"| SVR
    FS -->|"features"| LSTM

    LR -->|"predictions"| Eval
    SVR -->|"predictions"| Eval
    LSTM -->|"predictions"| Eval

    Eval -->|"results"| Viz

    %% Click Events
    click MNIST "https://github.com/lldelduca/btc_predictions_denoisingautoencoders/blob/main/Full_Notebook.ipynb"
    click Finance "https://github.com/lldelduca/btc_predictions_denoisingautoencoders/blob/main/Full_Notebook.ipynb"
    click Preproc "https://github.com/lldelduca/btc_predictions_denoisingautoencoders/blob/main/Full_Notebook.ipynb"
    click SAE "https://github.com/lldelduca/btc_predictions_denoisingautoencoders/blob/main/Full_Notebook.ipynb"
    click SDAE "https://github.com/lldelduca/btc_predictions_denoisingautoencoders/blob/main/Full_Notebook.ipynb"
    click FS "https://github.com/lldelduca/btc_predictions_denoisingautoencoders/blob/main/Full_Notebook.ipynb"
    click LR "https://github.com/lldelduca/btc_predictions_denoisingautoencoders/blob/main/Full_Notebook.ipynb"
    click SVR "https://github.com/lldelduca/btc_predictions_denoisingautoencoders/blob/main/Full_Notebook.ipynb"
    click LSTM "https://github.com/lldelduca/btc_predictions_denoisingautoencoders/blob/main/Full_Notebook.ipynb"
    click Eval "https://github.com/lldelduca/btc_predictions_denoisingautoencoders/blob/main/Full_Notebook.ipynb"
    click Viz "https://github.com/lldelduca/btc_predictions_denoisingautoencoders/blob/main/Full_Notebook.ipynb"
    click README "https://github.com/lldelduca/btc_predictions_denoisingautoencoders/blob/main/README.md"
    click LICENSE "https://github.com/lldelduca/btc_predictions_denoisingautoencoders/tree/main/LICENSE"

    %% Styles
    classDef data fill:#AED6F1,stroke:#1B4F72,color:#1B2631;
    classDef process fill:#ABEBC6,stroke:#1D8348,color:#145A32;
    classDef model fill:#F9E79F,stroke:#B7950B,color:#7D6608;
    classDef decision fill:#F5B7B1,stroke:#943126,color:#641E16;
```
