How soyamkapoor/10_pearls_aqi_predict_bot works
This is a Sukkur air-quality monitoring and forecasting product: OpenWeather observations and historical backfill feed local data and feature preparation, while Hopsworks stores features for model training. The Streamlit dashboard presents live conditions, pollutant views, historical charts, and three-day forecasts; separate scripts support alerts and model analysis. The main workflow is documented in the README and partly visible in the sampled source. Dashboard internals and several automation details were not fully sampled, so their links are limited to documented relationships.
Main entry points: streamlit_app.py Streamlit dashboard · cleaned_data.csv Cleaned dataset · train_model.py Model comparison · feature_engineering.py Feature engineering
Components 20
Data Ingestion
- Current data fetch
src/fetch_data.py - Historical backfill
src/backfill.py
Features And Models
- Feature engineering
src/feature_engineering.py - Cleaned dataset
data/cleaned_data.csv - Model comparison
src/train_model.py - Fast model training
src/train_model_fast.py - MLP training
src/train_mlp.py - Model persistence
model_registry/resgistry_utils.py - Shared configuration
src/utils.py
Monitoring And Forecasting
- Streamlit dashboard
dashboard/streamlit_app.py - AQI alerts
src/aqi_alerts.py - Offline prediction
src/predict_offline.py
Analysis And Alerts
- Exploratory analysis
src/eda_analysis.py - Result analysis
src/analyze_results.py - Model explanations
src/explain.py - Feature importance
src/feature_importance.py
Other components
- Dashboard user
- Scheduled updates
- OpenWeather API
- Hopsworks Feature Store
Connections 20
- Current data fetch → OpenWeather API: fetches readings
src/fetch_data.py - Historical backfill → OpenWeather API: fetches history
src/backfill.py - Feature engineering → Shared configuration: loads settings
src/feature_engineering.py - Feature engineering → Cleaned dataset: writes dataset
src/feature_engineering.py - Feature engineering → Hopsworks Feature Store: inserts features
src/feature_engineering.py - Model comparison → Shared configuration: loads settings
src/train_model.py - Model comparison → Hopsworks Feature Store: reads features
src/train_model.py - Fast model training → Shared configuration: loads settings
src/train_model_fast.py - Fast model training → Hopsworks Feature Store: reads features
src/train_model_fast.py - MLP training → Cleaned dataset: reads dataset
src/train_mlp.py - Streamlit dashboard → OpenWeather API: shows live data
README.md - Streamlit dashboard → Hopsworks Feature Store: uses feature store
README.md - Dashboard user → Streamlit dashboard: views dashboard
README.md - Streamlit dashboard → Model comparison: forecasts with ML
README.md - Scheduled updates → Streamlit dashboard: updates dashboard data
README.md - AQI alerts → Cleaned dataset: reads latest data
src/aqi_alerts.py - Offline prediction → Cleaned dataset: uses data
- Feature importance → Cleaned dataset: reads dataset
src/feature_importance.py - Feature importance → Model comparison: loads saved model
- Model persistence → Hopsworks Feature Store: stores models