# r-zeeshan/bit-analytica architecture

> Architecture diagram of the GitHub repository r-zeeshan/bit-analytica, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-05-17). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/r-zeeshan/bit-analytica
- Repository: https://github.com/r-zeeshan/bit-analytica

## Overview

##### Project type and purpose
- BitAnalytica is a full-stack data-driven application built with Python and Streamlit.
- Its purpose is to ingest and preprocess both Bitcoin pricing data and news text data, perform sentiment analysis, compute technical indicators, run ML inference (XGBoost, GBM, LSTM, TabNet), and present interactive dashboards and forecasts via a web UI.

##### File structure
- Top-level scripts:
- app.py – Streamlit entrypoint (frontend & coordinator).
- btc_data_pipeline.py, text_data_pipeline.py, data_scrapper.py – ETL modules.
- preprocess_*.ipynb, models.ipynb – exploratory & training notebooks (not in production flow).
- app_utils.py, btc_utils.py, text_utils.py, plot_utils.py – reusable utility functions.
- config.py – central configuration (API keys, file paths, model paths).
- Data storage:
- data/ – CSV files: raw and processed BTC data, sentiment scores, predictions.
- models/ – persisted model artifacts (high.keras, low.zip) and scalers.
- Dependencies: requirements.txt, external APIs (Yahoo Finance via yfinance, GDELT API, Hugging Face finbert-tone).

##### Insights from README
- Two ETL streams:
1. Bitcoin data: download historical OHLCV → calculate SMA, EMA, RSI, MACD, etc.
2. News data: GDELT scrape → clean & tokenize → BERT sentiment → aggregate daily scores.
- ML models trained offline; inference served in app.
- UI sections: hourly technicals, daily charts, prediction view.

##### Elements
- Data flow: External API → ETL → Data Store → Inference → UI
- Control flow: User → UI → backend modules
- Blue for UI components
- Green for ETL/data pipelines
- Yellow for storage (CSV, models/)
- Orange for external services

## Diagram (Mermaid)

```mermaid
flowchart TD
    %% Layer 1: Client
    User["User / Browser"]:::client

    %% Layer 2: Presentation Layer
    subgraph "Presentation Layer"
        direction TB
        Streamlit["Streamlit App\n(app.py)"]:::ui
        AppUtils["app_utils.py"]:::ui
    end

    %% Configuration Module
    Config["config.py"]:::config

    %% Layer 3: Plotting Layer
    subgraph "Plotting Layer"
        direction TB
        PlotUtils["plot_utils.py"]:::vis
    end

    %% Layer 4: ETL Layer
    subgraph "ETL Layer"
        direction TB
        BTC_Pipeline["BTC Data Pipeline\n(btc_data_pipeline.py)"]:::etl
        BTC_Utils["btc_utils.py"]:::etl
        News_Scraper["News Scraper\n(data_scrapper.py)"]:::etl
        Text_Pipeline["Text Data Pipeline\n(text_data_pipeline.py)"]:::etl
        Text_Utils["text_utils.py"]:::etl
    end

    %% External Services
    subgraph "External Services"
        direction TB
        Yahoo["Yahoo Finance API"]:::external
        GDELT["GDELT API"]:::external
        HuggingFace["HuggingFace finbert-tone"]:::external
    end

    %% Layer 5: Data Store
    subgraph "Data Store (CSV Files)"
        direction TB
        BTC_CSV["btc_data.csv"]:::storage
        Final_CSV["final_data.csv"]:::storage
        Pred_CSV["predictions.csv"]:::storage
        Sent_CSV["sentiment_scores.csv"]:::storage
    end

    %% Layer 6: ML Inference Layer
    subgraph "ML Inference Layer"
        direction TB
        HighModel["high.keras"]:::model
        LowModel["low.zip"]:::model
        XScaler["x_scaler.pkl"]:::model
        YHighScaler["y_high_scaler.pkl"]:::model
        YLowScaler["y_low_scaler.pkl"]:::model
    end

    %% Offline Notebooks & Manifest
    subgraph "Notebooks (Offline) & Manifest"
        direction TB
        Notebook1["preprocess_btc_data.ipynb"]:::notebook
        Notebook2["preprocess_news_data.ipynb"]:::notebook
        Notebook3["models.ipynb"]:::notebook
        Requirements["requirements.txt"]:::manifest
    end

    %% Connections: Control & Data Flows
    User -->|opens| Streamlit
    Streamlit -->|uses| AppUtils
    Streamlit -->|loads config| Config
    Streamlit -->|invokes plotting| PlotUtils
    AppUtils -->|runs| BTC_Pipeline
    AppUtils -->|runs| Text_Pipeline
    AppUtils -->|loads model| HighModel
    AppUtils -->|loads model| LowModel
    AppUtils -->|loads scaler| XScaler
    AppUtils -->|loads scaler| YHighScaler
    AppUtils -->|loads scaler| YLowScaler
    Streamlit -->|reads| BTC_CSV
    Streamlit -->|reads| Final_CSV
    Streamlit -->|reads| Sent_CSV
    Streamlit -->|reads/writes| Pred_CSV

    BTC_Pipeline -->|fetches| Yahoo
    BTC_Pipeline -->|uses| BTC_Utils
    BTC_Pipeline -->|writes| BTC_CSV

    News_Scraper -->|fetches| GDELT
    News_Scraper -->|uses| Text_Utils
    News_Scraper -->|writes| Sent_CSV

    Text_Pipeline -->|fetches| HuggingFace
    Text_Pipeline -->|uses| Text_Utils
    Text_Pipeline -->|writes| Final_CSV

    HighModel -->|in inference| Pred_CSV
    LowModel -->|in inference| Pred_CSV
    XScaler -->|in inference| Pred_CSV
    YHighScaler -->|in inference| Pred_CSV
    YLowScaler -->|in inference| Pred_CSV

    %% Click Events
    click Streamlit "https://github.com/r-zeeshan/bit-analytica/blob/main/app.py"
    click AppUtils "https://github.com/r-zeeshan/bit-analytica/blob/main/app_utils.py"
    click Config "https://github.com/r-zeeshan/bit-analytica/blob/main/config.py"
    click PlotUtils "https://github.com/r-zeeshan/bit-analytica/blob/main/plot_utils.py"
    click BTC_Pipeline "https://github.com/r-zeeshan/bit-analytica/blob/main/btc_data_pipeline.py"
    click BTC_Utils "https://github.com/r-zeeshan/bit-analytica/blob/main/btc_utils.py"
    click News_Scraper "https://github.com/r-zeeshan/bit-analytica/blob/main/data_scrapper.py"
    click Text_Pipeline "https://github.com/r-zeeshan/bit-analytica/blob/main/text_data_pipeline.py"
    click Text_Utils "https://github.com/r-zeeshan/bit-analytica/blob/main/text_utils.py"
    click BTC_CSV "https://github.com/r-zeeshan/bit-analytica/blob/main/data/btc_data.csv"
    click Final_CSV "https://github.com/r-zeeshan/bit-analytica/blob/main/data/final_data.csv"
    click Pred_CSV "https://github.com/r-zeeshan/bit-analytica/blob/main/data/predictions.csv"
    click Sent_CSV "https://github.com/r-zeeshan/bit-analytica/blob/main/data/sentiment_scores.csv"
    click HighModel "https://github.com/r-zeeshan/bit-analytica/blob/main/models/high/high.keras"
    click LowModel "https://github.com/r-zeeshan/bit-analytica/blob/main/models/low/low.zip"
    click XScaler "https://github.com/r-zeeshan/bit-analytica/blob/main/models/scalers/x_scaler.pkl"
    click YHighScaler "https://github.com/r-zeeshan/bit-analytica/blob/main/models/scalers/y_high_scaler.pkl"
    click YLowScaler "https://github.com/r-zeeshan/bit-analytica/blob/main/models/scalers/y_low_scaler.pkl"

    %% Styles
    classDef client fill:#8dd3c7,stroke:#000
    classDef ui fill:#80b1d3,stroke:#000
    classDef config fill:#bebada,stroke:#000
    classDef etl fill:#b3de69,stroke:#000
    classDef storage fill:#ffffb3,stroke:#000
    classDef model fill:#fb8072,stroke:#000
    classDef external fill:#fdb462,stroke:#000
    classDef notebook fill:#fccde5,stroke:#000
    classDef manifest fill:#d9d9d9,stroke:#000
    classDef vis fill:#66c2a5,stroke:#000
```
