# amulyaprasanth/hourly_amazon_stock_prediction architecture

> Architecture diagram of the GitHub repository amulyaprasanth/hourly_amazon_stock_prediction, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-07-01). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/amulyaprasanth/hourly_amazon_stock_prediction
- Repository: https://github.com/amulyaprasanth/hourly_amazon_stock_prediction

## Overview

##### Project Type & Purpose
- This is a full-stack machine-learning application that:
- Ingests historical Amazon stock data
- Runs feature-engineering and model-training pipelines
- Executes batch inference on an hourly cadence
- Exposes results via a lightweight dashboard (Streamlit)
- Uses GitHub Actions to automate pipeline runs and dashboard deployment

##### Key Components (Based on File Structure)
a. Data & Configuration
- External Data Source (e.g., AlphaVantage / Yahoo Finance API)
- Local/Cloud Storage for raw data & feature store (not explicit in code but implied)
- config/config.yml
b. Feature Engineering
- amazonstockprediction/feature_pipeline.py
- amazonstockprediction/backfill_features.py
c. Model Training
- amazonstockprediction/training_pipeline.py
d. Batch Inference
- amazonstockprediction/batch_inference_pipeline.py
e. Logging & Utilities
- amazonstockprediction/logger.py
- amazonstockprediction/utils.py
f. Dashboard
- dashboard/app.py
- dashboard/requirements.txt
g. Orchestration & CI/CD
- .github/workflows/train_pipeline.yml
- .github/workflows/feature_and_inference_pipeline.yml
- .github/workflows/reboot_streamlit.yml
h. Notebooks & Tests
- notebooks/*.ipynb (exploratory & demo)
- tests/test_utils.py

##### Architectural Patterns & Relationships
- **Pipelines as Modular Blocks:** Each pipeline (features → training → inference) is a self-contained Python script.
- **Sequential Data Flow:** Raw data → feature store → model training → model artifacts → batch inference → predictions storage → dashboard.
- **Orchestration Layer:** GitHub Actions schedules and triggers each pipeline; also handles dashboard (re)deployment.
- **Config-Driven:** Shared config.yml drives parameters for all pipelines, ensuring consistency.
- **Testing:** A lightweight test suite validates utilities.

##### Detailing Each Component
- **Feature Engineering:**
- backfill_features.py (initial backfill)
- feature_pipeline.py (hourly updates)
- **Training Pipeline:**
- training_pipeline.py (consumes latest features → outputs model artifact)
- **Batch Inference:**
- batch_inference_pipeline.py (loads model → generates predictions)
- **Dashboard:**
- app.py (Streamlit; pulls predictions and visualizes)
- **Orchestration:**
- GitHub Actions workflows calling Python modules on schedule or event
- **Configuration:**
- config.yml (single source of truth for database paths, API keys, model hyperparameters)

## Diagram (Mermaid)

```mermaid
flowchart TB
    subgraph "External Data & Configuration"
        extAPI["External Data Source (AlphaVantage/Yahoo)"]:::data
        config["config/config.yml"]:::data
    end

    subgraph "Feature Engineering"
        backfill["backfill_features.py"]:::pipeline
        featurePipe["feature_pipeline.py"]:::pipeline
    end

    subgraph "Model Training"
        train["training_pipeline.py"]:::pipeline
    end

    subgraph "Batch Inference"
        infer["batch_inference_pipeline.py"]:::pipeline
    end

    subgraph "Logging & Utilities"
        logger["logger.py"]:::pipeline
        utils["utils.py"]:::pipeline
    end

    subgraph "Dashboard"
        dash["Dashboard UI (app.py)"]:::ui
        deps["requirements.txt"]:::data
    end

    subgraph "CI/CD & Orchestration"
        ciFeatInfer["feature_and_inference_pipeline.yml"]:::ci
        ciTrain["train_pipeline.yml"]:::ci
        ciDash["reboot_streamlit.yml"]:::ci
    end

    subgraph "Notebooks & Tests"
        notebooks["Notebooks"]:::aux
        tests["tests/test_utils.py"]:::aux
    end

    extAPI -->|"Raw Data (CSV)"| backfill
    config --> backfill
    backfill -->|"Backfilled Features (CSV)"| featurePipe
    featurePipe -->|"Features (CSV)"| train
    train -->|"Model Artifact (Pickle)"| infer
    infer -->|"Predictions (JSON)"| dash

    ciFeatInfer --> featurePipe
    ciFeatInfer --> infer
    ciTrain --> train
    ciDash --> dash

    notebooks -.-> backfill
    notebooks -.-> featurePipe
    notebooks -.-> train
    notebooks -.-> infer
    tests -.-> utils

    classDef data fill:#ccf,stroke:#333,stroke-width:1px
    classDef pipeline fill:#cfc,stroke:#333,stroke-width:1px
    classDef ui fill:#fcf,stroke:#333,stroke-width:1px
    classDef ci fill:#ccf,stroke:#333,stroke-width:1px
    classDef aux stroke-dasharray: 5 5,fill:#fff

    click config "https://github.com/amulyaprasanth/hourly_amazon_stock_prediction/blob/main/config/config.yml"
    click backfill "https://github.com/amulyaprasanth/hourly_amazon_stock_prediction/blob/main/amazonstockprediction/backfill_features.py"
    click featurePipe "https://github.com/amulyaprasanth/hourly_amazon_stock_prediction/blob/main/amazonstockprediction/feature_pipeline.py"
    click train "https://github.com/amulyaprasanth/hourly_amazon_stock_prediction/blob/main/amazonstockprediction/training_pipeline.py"
    click infer "https://github.com/amulyaprasanth/hourly_amazon_stock_prediction/blob/main/amazonstockprediction/batch_inference_pipeline.py"
    click logger "https://github.com/amulyaprasanth/hourly_amazon_stock_prediction/blob/main/amazonstockprediction/logger.py"
    click utils "https://github.com/amulyaprasanth/hourly_amazon_stock_prediction/blob/main/amazonstockprediction/utils.py"
    click dash "https://github.com/amulyaprasanth/hourly_amazon_stock_prediction/blob/main/dashboard/app.py"
    click deps "https://github.com/amulyaprasanth/hourly_amazon_stock_prediction/blob/main/dashboard/requirements.txt"
    click ciFeatInfer "https://github.com/amulyaprasanth/hourly_amazon_stock_prediction/blob/main/.github/workflows/feature_and_inference_pipeline.yml"
    click ciTrain "https://github.com/amulyaprasanth/hourly_amazon_stock_prediction/blob/main/.github/workflows/train_pipeline.yml"
    click ciDash "https://github.com/amulyaprasanth/hourly_amazon_stock_prediction/blob/main/.github/workflows/reboot_streamlit.yml"
    click notebooks "https://github.com/amulyaprasanth/hourly_amazon_stock_prediction/tree/main/notebooks/"
    click tests "https://github.com/amulyaprasanth/hourly_amazon_stock_prediction/blob/main/tests/test_utils.py"
```
