# anvayb/skystream architecture

> Architecture diagram of the GitHub repository anvayb/skystream, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-05-22). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/anvayb/skystream
- Repository: https://github.com/anvayb/skystream

## Overview

##### Project Type and Purpose
- SkyStream is a data-driven analytics and prediction platform (not a traditional multi-tier web app).
- Core functions:
- Batch data ingestion & exploration (EDA & Models notebooks)
- Machine-learning model training & serving (delay predictor)
- Interactive dashboards for visualization (Streamlit app)

##### File Structure
Top-level:
- “EDA & Models/” – Jupyter notebook for exploratory data analysis and model development.
- “Streamlit/” – Python code (`Visualization Dashboard.py`), notebooks (`Probability Predictor.ipynb`), PDFs of intended UI mockups.
- `dataset.txt` – pointer to raw datasets.
- README & project presentation PDFs.
No separate “frontend” or “backend” folders—instead, Streamlit combines front-end UI and Python code.

##### README for Architecture Insights
- Data sources: Combined Flights 2021/2022 CSVs (Kaggle).
- Dependencies: Python 3.8+, PySpark, Streamlit, scikit-learn (inferred).
- Runtime: Local (Flask/Streamlit on port 5000).

##### Main Components
a. Data Sources
- External CSV datasets (Kaggle)
- (Optionally) Real-time feed or API for weather data
b. Data Ingestion & Processing
- PySpark jobs or Pandas pipelines in notebooks
- ETL transformations, feature engineering
c. Model Training & Storage
- Jupyter environment where ML models are trained
- Model artifacts persisted (Pickle, MLflow, etc.)
d. Model Serving / Prediction Engine
- Python module (Probability Predictor.ipynb / package)
- Exposes prediction API or function calls
e. Visualization Dashboard (Streamlit App)
- Single-page UI combining:
– Flight Delay Dashboard (maps, charts)
– Delay Predictor form & output
- Hosted locally (Flask/Streamlit server on :5000)

##### Relationships & Data Flows
- Raw CSVs → (1) EDA & Model Training
- Feature store & model artifacts → (2) Deployed to the Prediction Engine
- User events (UI inputs) → Prediction Engine → UI update
- Periodic batch jobs → refresh datasets or model retraining

##### Architectural Patterns & Design Principles
- Modular separation:
- Batch (EDA & training) vs. real-time (prediction & visualization)
- Reusable ML component: training pipeline decoupled from serving code
- Single page app: Streamlit functions as both UI and lightweight backend

##### Final Checklist
- Components: Data Sources, ETL/Processing, Model Training, Model Store, Prediction Engine, Dashboard
- Labels: component names + key frameworks

## Diagram (Mermaid)

```mermaid
flowchart TD
    %% Data Components
    DataSources["Data Sources\n(CSVs)"]:::data
    WeatherAPI["Weather API\n(Optional)"]:::data

    %% Compute Components
    ETL["ETL & Processing\n(PySpark)"]:::compute
    ModelTraining["Model Training\n(Jupyter / scikit-learn)"]:::compute
    PredictionEngine["Prediction Engine\n(Streamlit / Python)"]:::compute

    %% Storage
    ModelStore["Model Store\n(Pickle / MLflow)"]:::data

    %% UI Components
    VisualizationDashboard["Visualization Dashboard\n(Streamlit UI)"]:::ui

    %% Documentation
    UI_Mockups["UI Mockups\n(PDFs)"]:::docs
    Docs["Project Docs\n(README & Presentation)"]:::docs

    %% Flows
    DataSources -->|Raw CSVs| ETL
    WeatherAPI -->|Weather Data| ETL
    ETL -->|Features| ModelTraining
    ModelTraining -->|Model Artifacts| ModelStore
    ModelStore -->|Load Models| PredictionEngine
    VisualizationDashboard -->|User Inputs| PredictionEngine
    PredictionEngine -->|Predictions| VisualizationDashboard

    %% Side Documentation Links
    VisualizationDashboard --> UI_Mockups
    Docs --> ETL
    Docs --> VisualizationDashboard

    %% Click Events
    click DataSources "https://github.com/anvayb/skystream/blob/main/dataset.txt"
    click ETL "https://github.com/anvayb/skystream/blob/main/EDA & Models/Group 6 - SkyStream EDA & Models code.ipynb"
    click ModelTraining "https://github.com/anvayb/skystream/blob/main/EDA & Models/Group 6 - SkyStream EDA & Models code.ipynb"
    click PredictionEngine "https://github.com/anvayb/skystream/blob/main/Streamlit/Probability Predictor.ipynb"
    click VisualizationDashboard "https://github.com/anvayb/skystream/blob/main/Streamlit/Visualization Dashboard.py"
    click UI_Mockups "https://github.com/anvayb/skystream/blob/main/Streamlit/FlightDelayDashboard1.pdf"
    click UI_Mockups "https://github.com/anvayb/skystream/blob/main/Streamlit/FlightDelayDashboard2.pdf"
    click UI_Mockups "https://github.com/anvayb/skystream/blob/main/Streamlit/FlightDelayPredictor1.pdf"
    click UI_Mockups "https://github.com/anvayb/skystream/blob/main/Streamlit/FlightDelayPredictor2.pdf"
    click Docs "https://github.com/anvayb/skystream/blob/main/README.md"
    click Docs "https://github.com/anvayb/skystream/blob/main/SkyStream Project Presentation.pdf"

    %% Styles
    classDef data fill:#BBDEFB,stroke:#1E88E5,color:#0D47A1
    classDef compute fill:#C8E6C9,stroke:#43A047,color:#1B5E20
    classDef ui fill:#FFE0B2,stroke:#FB8C00,color:#E65100
    classDef docs fill:#E1BEE7,stroke:#8E24AA,color:#4A148C
```
