# manish-kotra/algorithmic-trading architecture

> Architecture diagram of the GitHub repository manish-kotra/algorithmic-trading, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-03-31). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/manish-kotra/algorithmic-trading
- Repository: https://github.com/manish-kotra/algorithmic-trading

## Overview

##### Main Components (the “modules”
a. Data Collection Layer:
- External Data Sources: Represent external services and websites/web pages where data is obtained, such as bitcoininfocharts, finviz, and Google News.
- Webscraping Notebooks: Include components like “webscrapper_bitcoinfocharts.ipynb”, “webscrapper_finviz.ipynb”, and “webscrapper_googlenews.ipynb” which collect raw data.

b. Data Preprocessing & Feature Engineering Layer:
- Data Cleaning/Preprocessing: Represent notebooks or scripts such as “bitcoin_feature_engineering.ipynb” and possibly “detrendPrice.py”, which prepare and normalize data.
- Data Consolidation: Show how processed datasets (for example, “bitcoin_techindicators_filtered_normalized.csv” and “main.csv”) flow to the next stage.

c. Sentiment Analysis Layer:
- VADER Sentiment Analysis: A module for “news_sentiment_analysis_vader.ipynb” that processes news headlines using predefined sentiment rules.
- BERT Sentiment Analysis: A module for “news_sentiment_analysis_bert.ipynb” that leverages transformer models, outputting sentiment metrics.
- Data Storage: CSVs from sentiment analysis (like “bitcoin_news_data_sentiment_vader.csv” and “bitcoin_news_data_sentiment_finbert.csv”) should be depicted as outputs of these processes.

d. Exploratory and Correlation Analysis:
- EDA and Visualization: Include “bitcoin_tsa.ipynb” and “news_btc_correlation.ipynb” which facilitate exploratory data analysis and establish relationships between market data and sentiment.
- Results and Artifacts: Highlight generated insights such as “grid_search_results.pkl” (for model tuning) and result figures (from the figures folder).

e. Model Tuning and Strategy Evaluation:

##### Relationships and Data Flow
- That processed data (via feature engineering) and sentiment results are merged or passed together into model training and evaluation steps.

##### Key Architectural Patterns & Design Principles
- Data Pipeline: Emphasize the stepwise flow of data – collection, cleaning/processing, feature engineering, analytics, and final evaluations.
- Modularity: Illustrate how each notebook or script represents a distinct functional component. This separation of concerns aids in maintenance and further extension.
- Reproducibility & Environment: Mention the “environment.yml” which ensures reproducible builds; this underpins the technical stack (Python, conda environment).

## Diagram (Mermaid)

```mermaid
graph TB
    %% External Data Sources
    subgraph "External Data Sources"
        external_bitcoin("BitcoinInfoCharts"):::external
        external_finviz("Finviz"):::external
        external_google("Google News"):::external
    end

    %% Data Collection Layer
    subgraph "Data Collection Layer"
        ws_bitcoinfocharts("webscrapper_bitcoinfocharts.ipynb"):::module
        ws_finviz("webscrapper_finviz.ipynb"):::module
        ws_googlenews("webscrapper_googlenews.ipynb"):::module
        csv_data("CSV Files"):::datastore
    end

    %% Data Preprocessing & Feature Engineering Layer
    subgraph "Data Preprocessing & Feature Engineering Layer"
        df_engineering("bitcoin_feature_engineering.ipynb"):::module
        detrend("detrendPrice.py"):::module
        processed1("bitcoin_techindicators_filtered_normalized.csv"):::datastore
        processed2("main.csv"):::datastore
    end

    %% Sentiment Analysis Layer
    subgraph "Sentiment Analysis Layer"
        vader("news_sentiment_analysis_vader.ipynb"):::module
        bert("news_sentiment_analysis_bert.ipynb"):::module
        sentiment_vader("bitcoin_news_data_sentiment_vader.csv"):::datastore
        sentiment_bert("bitcoin_news_data_sentiment_finbert.csv"):::datastore
    end

    %% Exploratory and Correlation Analysis
    subgraph "Exploratory and Correlation Analysis"
        eda_btc("bitcoin_tsa.ipynb"):::module
        eda_corr("news_btc_correlation.ipynb"):::module
        artifacts_fig("figures"):::datastore
        grid_search("grid_search_results.pkl"):::datastore
    end

    %% Model Tuning and Strategy Evaluation
    subgraph "Model Tuning and Strategy Evaluation"
        main_eval("main.ipynb"):::module
        aux_faux("fAux.py"):::module
        aux_check("WhiteRealityCheckFor1.py"):::module
    end

    %% Environment & Reproducibility
    subgraph "Environment & Reproducibility"
        env("environment.yml"):::module
    end

    %% Data Flow Connections
    %% External Sources to Data Collection
    external_bitcoin --> ws_bitcoinfocharts
    external_finviz --> ws_finviz
    external_google --> ws_googlenews

    %% Data Collection to CSV Data Storage
    ws_bitcoinfocharts --> csv_data
    ws_finviz --> csv_data
    ws_googlenews --> csv_data

    %% CSV Data flows to Preprocessing
    csv_data --> df_engineering
    csv_data --> detrend

    df_engineering --> processed1
    detrend --> processed2

    %% CSV Data flows into Sentiment Analysis
    csv_data --> vader
    csv_data --> bert

    vader --> sentiment_vader
    bert --> sentiment_bert

    %% Processed Data and Sentiment Results feed into EDA
    processed1 --> eda_btc
    processed2 --> eda_btc
    sentiment_vader --> eda_corr
    sentiment_bert --> eda_corr

    eda_btc --> artifacts_fig
    eda_corr --> grid_search

    %% EDA and Processed Data feed into Main Evaluation
    processed1 --> main_eval
    sentiment_vader --> main_eval
    sentiment_bert --> main_eval
    eda_btc --> main_eval
    eda_corr --> main_eval

    %% Auxiliary functions assist Main Evaluation
    aux_faux --> main_eval
    aux_check --> main_eval

    %% Environment supports Main Evaluation
    env --> main_eval

    %% Styles
    classDef external fill:#ffcccb,stroke:#333,stroke-width:2px;
    classDef module fill:#bbf,stroke:#333,stroke-width:2px;
    classDef datastore fill:#cfc,stroke:#333,stroke-width:2px;

click ws_bitcoinfocharts "https://github.com/manish-kotra/algorithmic-trading/blob/main/webscrapper_bitcoinfocharts.ipynb"
click ws_finviz "https://github.com/manish-kotra/algorithmic-trading/blob/main/webscrapper_finviz.ipynb"
click ws_googlenews "https://github.com/manish-kotra/algorithmic-trading/blob/main/webscrapper_googlenews.ipynb"
click csv_data "https://github.com/manish-kotra/algorithmic-trading/tree/main/csv_files"
click df_engineering "https://github.com/manish-kotra/algorithmic-trading/blob/main/bitcoin_feature_engineering.ipynb"
click detrend "https://github.com/manish-kotra/algorithmic-trading/blob/main/detrendPrice.py"
click processed1 "https://github.com/manish-kotra/algorithmic-trading/blob/main/csv_files/bitcoin_techindicators_filtered_normalized.csv"
click processed2 "https://github.com/manish-kotra/algorithmic-trading/blob/main/csv_files/main.csv"
click vader "https://github.com/manish-kotra/algorithmic-trading/blob/main/news_sentiment_analysis_vader.ipynb"
click bert "https://github.com/manish-kotra/algorithmic-trading/blob/main/news_sentiment_analysis_bert.ipynb"
click sentiment_vader "https://github.com/manish-kotra/algorithmic-trading/blob/main/csv_files/bitcoin_news_data_sentiment_vader.csv"
click sentiment_bert "https://github.com/manish-kotra/algorithmic-trading/blob/main/csv_files/bitcoin_news_data_sentiment_finbert.csv"
click eda_btc "https://github.com/manish-kotra/algorithmic-trading/blob/main/bitcoin_tsa.ipynb"
click eda_corr "https://github.com/manish-kotra/algorithmic-trading/blob/main/news_btc_correlation.ipynb"
click artifacts_fig "https://github.com/manish-kotra/algorithmic-trading/tree/main/figures"
click grid_search "https://github.com/manish-kotra/algorithmic-trading/blob/main/grid_search_results.pkl"
click main_eval "https://github.com/manish-kotra/algorithmic-trading/blob/main/main.ipynb"
click aux_faux "https://github.com/manish-kotra/algorithmic-trading/blob/main/fAux.py"
click aux_check "https://github.com/manish-kotra/algorithmic-trading/blob/main/WhiteRealityCheckFor1.py"
click env "https://github.com/manish-kotra/algorithmic-trading/blob/main/environment.yml"
```
