# soyamkapoor/10_pearls_aqi_predict_bot architecture

> Architecture diagram of the GitHub repository soyamkapoor/10_pearls_aqi_predict_bot, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-10-06). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/soyamkapoor/10_pearls_aqi_predict_bot
- Repository: https://github.com/soyamkapoor/10_pearls_aqi_predict_bot

## Overview

This is a Sukkur air-quality monitoring and forecasting product: OpenWeather observations and historical backfill feed local data and feature preparation, while Hopsworks stores features for model training. The Streamlit dashboard presents live conditions, pollutant views, historical charts, and three-day forecasts; separate scripts support alerts and model analysis. The main workflow is documented in the README and partly visible in the sampled source. Dashboard internals and several automation details were not fully sampled, so their links are limited to documented relationships.

Main entry points: [`dashboard/streamlit_app.py`](https://github.com/soyamkapoor/10_pearls_aqi_predict_bot/blob/main/dashboard/streamlit_app.py) (Streamlit dashboard), [`data/cleaned_data.csv`](https://github.com/soyamkapoor/10_pearls_aqi_predict_bot/blob/main/data/cleaned_data.csv) (Cleaned dataset), [`src/train_model.py`](https://github.com/soyamkapoor/10_pearls_aqi_predict_bot/blob/main/src/train_model.py) (Model comparison), [`src/feature_engineering.py`](https://github.com/soyamkapoor/10_pearls_aqi_predict_bot/blob/main/src/feature_engineering.py) (Feature engineering)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_ingest["Data Ingestion"]
  node_current_fetch["Current data fetch<br/>[fetch_data.py]"]
  node_historical_backfill["Historical backfill<br/>[backfill.py]"]
end

subgraph group_ml["Features And Models"]
  node_feature_engineering["Feature engineering"]
  node_cleaned_data[("Cleaned dataset<br/>[cleaned_data.csv]")]
  node_training["Model comparison<br/>[train_model.py]"]
  node_fast_training["Fast model training"]
  node_mlp_training["MLP training<br/>[train_mlp.py]"]
  node_model_registry["Model persistence<br/>[resgistry_utils.py]"]
  node_config_utils["Shared configuration<br/>[utils.py]"]
end

subgraph group_serve["Monitoring And Forecasting"]
  node_dashboard["Streamlit dashboard<br/>[streamlit_app.py]"]
  node_alerts["AQI alerts<br/>[aqi_alerts.py]"]
  node_offline_prediction["Offline prediction<br/>[predict_offline.py]"]
end

subgraph group_insights["Analysis And Alerts"]
  node_eda["Exploratory analysis<br/>[eda_analysis.py]"]
  node_results_analysis["Result analysis<br/>[analyze_results.py]"]
  node_explainability["Model explanations<br/>[explain.py]"]
  node_feature_importance["Feature importance"]
end

node_user(("Dashboard user"))
node_automation(("Scheduled updates"))
node_openweather{{"OpenWeather API"}}
node_hopsworks[("Hopsworks Feature Store")]

node_current_fetch -->|"fetches readings"| node_openweather
node_historical_backfill -->|"fetches history"| node_openweather
node_feature_engineering -->|"loads settings"| node_config_utils
node_feature_engineering -->|"writes dataset"| node_cleaned_data
node_feature_engineering -->|"inserts features"| node_hopsworks
node_training -->|"loads settings"| node_config_utils
node_training -->|"reads features"| node_hopsworks
node_fast_training -->|"loads settings"| node_config_utils
node_fast_training -->|"reads features"| node_hopsworks
node_mlp_training -->|"reads dataset"| node_cleaned_data
node_dashboard -->|"shows live data"| node_openweather
node_dashboard -->|"uses feature store"| node_hopsworks
node_user -->|"views dashboard"| node_dashboard
node_dashboard -.->|"forecasts with ML"| node_training
node_automation -.->|"updates dashboard data"| node_dashboard
node_alerts -->|"reads latest data"| node_cleaned_data
node_offline_prediction -.->|"uses data"| node_cleaned_data
node_feature_importance -->|"reads dataset"| node_cleaned_data
node_feature_importance -.->|"loads saved model"| node_training
node_model_registry -.->|"stores models"| node_hopsworks

click node_current_fetch "https://github.com/soyamkapoor/10_pearls_aqi_predict_bot/blob/main/src/fetch_data.py"
click node_historical_backfill "https://github.com/soyamkapoor/10_pearls_aqi_predict_bot/blob/main/src/backfill.py"
click node_feature_engineering "https://github.com/soyamkapoor/10_pearls_aqi_predict_bot/blob/main/src/feature_engineering.py"
click node_cleaned_data "https://github.com/soyamkapoor/10_pearls_aqi_predict_bot/blob/main/data/cleaned_data.csv"
click node_training "https://github.com/soyamkapoor/10_pearls_aqi_predict_bot/blob/main/src/train_model.py"
click node_fast_training "https://github.com/soyamkapoor/10_pearls_aqi_predict_bot/blob/main/src/train_model_fast.py"
click node_mlp_training "https://github.com/soyamkapoor/10_pearls_aqi_predict_bot/blob/main/src/train_mlp.py"
click node_model_registry "https://github.com/soyamkapoor/10_pearls_aqi_predict_bot/blob/main/model_registry/resgistry_utils.py"
click node_dashboard "https://github.com/soyamkapoor/10_pearls_aqi_predict_bot/blob/main/dashboard/streamlit_app.py"
click node_alerts "https://github.com/soyamkapoor/10_pearls_aqi_predict_bot/blob/main/src/aqi_alerts.py"
click node_offline_prediction "https://github.com/soyamkapoor/10_pearls_aqi_predict_bot/blob/main/src/predict_offline.py"
click node_eda "https://github.com/soyamkapoor/10_pearls_aqi_predict_bot/blob/main/src/eda_analysis.py"
click node_results_analysis "https://github.com/soyamkapoor/10_pearls_aqi_predict_bot/blob/main/src/analyze_results.py"
click node_explainability "https://github.com/soyamkapoor/10_pearls_aqi_predict_bot/blob/main/src/explain.py"
click node_feature_importance "https://github.com/soyamkapoor/10_pearls_aqi_predict_bot/blob/main/src/feature_importance.py"
click node_config_utils "https://github.com/soyamkapoor/10_pearls_aqi_predict_bot/blob/main/src/utils.py"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_current_fetch,node_historical_backfill,node_user toneBlue
class node_feature_engineering,node_cleaned_data,node_training,node_fast_training,node_mlp_training,node_model_registry,node_config_utils,node_hopsworks toneAmber
class node_dashboard,node_alerts,node_offline_prediction,node_openweather toneMint
class node_eda,node_results_analysis,node_explainability,node_feature_importance toneRose
class node_automation toneIndigo
```

## Components

### Data Ingestion

- [Current data fetch](https://github.com/soyamkapoor/10_pearls_aqi_predict_bot/blob/main/src/fetch_data.py): `src/fetch_data.py`
- [Historical backfill](https://github.com/soyamkapoor/10_pearls_aqi_predict_bot/blob/main/src/backfill.py): `src/backfill.py`

### Features And Models

- [Feature engineering](https://github.com/soyamkapoor/10_pearls_aqi_predict_bot/blob/main/src/feature_engineering.py): `src/feature_engineering.py`
- [Cleaned dataset](https://github.com/soyamkapoor/10_pearls_aqi_predict_bot/blob/main/data/cleaned_data.csv): `data/cleaned_data.csv`
- [Model comparison](https://github.com/soyamkapoor/10_pearls_aqi_predict_bot/blob/main/src/train_model.py): `src/train_model.py`
- [Fast model training](https://github.com/soyamkapoor/10_pearls_aqi_predict_bot/blob/main/src/train_model_fast.py): `src/train_model_fast.py`
- [MLP training](https://github.com/soyamkapoor/10_pearls_aqi_predict_bot/blob/main/src/train_mlp.py): `src/train_mlp.py`
- [Model persistence](https://github.com/soyamkapoor/10_pearls_aqi_predict_bot/blob/main/model_registry/resgistry_utils.py): `model_registry/resgistry_utils.py`
- [Shared configuration](https://github.com/soyamkapoor/10_pearls_aqi_predict_bot/blob/main/src/utils.py): `src/utils.py`

### Monitoring And Forecasting

- [Streamlit dashboard](https://github.com/soyamkapoor/10_pearls_aqi_predict_bot/blob/main/dashboard/streamlit_app.py): `dashboard/streamlit_app.py`
- [AQI alerts](https://github.com/soyamkapoor/10_pearls_aqi_predict_bot/blob/main/src/aqi_alerts.py): `src/aqi_alerts.py`
- [Offline prediction](https://github.com/soyamkapoor/10_pearls_aqi_predict_bot/blob/main/src/predict_offline.py): `src/predict_offline.py`

### Analysis And Alerts

- [Exploratory analysis](https://github.com/soyamkapoor/10_pearls_aqi_predict_bot/blob/main/src/eda_analysis.py): `src/eda_analysis.py`
- [Result analysis](https://github.com/soyamkapoor/10_pearls_aqi_predict_bot/blob/main/src/analyze_results.py): `src/analyze_results.py`
- [Model explanations](https://github.com/soyamkapoor/10_pearls_aqi_predict_bot/blob/main/src/explain.py): `src/explain.py`
- [Feature importance](https://github.com/soyamkapoor/10_pearls_aqi_predict_bot/blob/main/src/feature_importance.py): `src/feature_importance.py`

### Other components

- **Dashboard user**
- **Scheduled updates**
- **OpenWeather API**
- **Hopsworks Feature Store**

## Connections

- Current data fetch → OpenWeather API: fetches readings (evidence: `src/fetch_data.py`)
- Historical backfill → OpenWeather API: fetches history (evidence: `src/backfill.py`)
- Feature engineering → Shared configuration: loads settings (evidence: `src/feature_engineering.py`)
- Feature engineering → Cleaned dataset: writes dataset (evidence: `src/feature_engineering.py`)
- Feature engineering → Hopsworks Feature Store: inserts features (evidence: `src/feature_engineering.py`)
- Model comparison → Shared configuration: loads settings (evidence: `src/train_model.py`)
- Model comparison → Hopsworks Feature Store: reads features (evidence: `src/train_model.py`)
- Fast model training → Shared configuration: loads settings (evidence: `src/train_model_fast.py`)
- Fast model training → Hopsworks Feature Store: reads features (evidence: `src/train_model_fast.py`)
- MLP training → Cleaned dataset: reads dataset (evidence: `src/train_mlp.py`)
- Streamlit dashboard → OpenWeather API: shows live data (evidence: `README.md`)
- Streamlit dashboard → Hopsworks Feature Store: uses feature store (evidence: `README.md`)
- Dashboard user → Streamlit dashboard: views dashboard (evidence: `README.md`)
- Streamlit dashboard → Model comparison: forecasts with ML (evidence: `README.md`)
- Scheduled updates → Streamlit dashboard: updates dashboard data (evidence: `README.md`)
- AQI alerts → Cleaned dataset: reads latest data (evidence: `src/aqi_alerts.py`)
- Offline prediction → Cleaned dataset: uses data
- Feature importance → Cleaned dataset: reads dataset (evidence: `src/feature_importance.py`)
- Feature importance → Model comparison: loads saved model
- Model persistence → Hopsworks Feature Store: stores models
