How arhamorioner/uhi-analysis works
Project Type and Purpose
• This is a data-analysis & machine-learning pipeline implemented in a single Python process (uhi_analysis.py), driven by user‐configurable parameters (config_params.py).
• Its purpose: fetch satellite data (via Google Earth Engine), preprocess it, detect and analyze Urban Heat Islands, train/predict LST with ML, generate plots and an interactive map.
File Structure
• Top-level files:
• config_params.py – all user parameters (city name, ROI, ML hyperparameters)
• uhi_analysis.py – orchestrates the entire pipeline
• requirements.txt – Python library dependencies (GEE client, pandas, scikit-learn, folium, SHAP, etc.)
• No persistent database: data is streamed from GEE, processed in memory, and outputs are saved as PNG/HTML in outputs/.
README for Additional Insights
• External services:
• Google Earth Engine API for multi-temporal satellite imagery
• Major pipeline stages explicitly documented:
1. Data Acquisition
2. Preprocessing & EDA
3. UHI Detection & Spatial Hotspot Analysis
4. Machine-Learning Model Training & Prediction
5. Model Interpretability (SHAP)
6. Visualization & Interactive Map Generation
Main Components & Interactions
a. Configuration Module (config_params.py)
• Defines: CITY_NAME, ROI_BOUNDS, years, seasons, resolution, ML hyperparams.
b. Data Ingestion Layer
• Google Earth Engine Client (external service)
• Optional Nominatim Client (external service, via find_city_roi.py)
c. Preprocessing & EDA
• Data sampling → Pandas DataFrame
• Cleaning (NaN handling), feature encoding, summary statistics, plots
d. UHI Detection & Spatial Analysis
• K-Means clustering for UHI zone classification
• Getis-Ord Gi* hot/cold spot identification
• UHI intensity calculation (urban vs. rural LST difference)
e. ML Prediction Module
• Random Forest pipeline, hyperparameter search (RandomizedSearchCV)
• Cross-validation, performance metrics
f. Model Interpretability
• SHAP explainer, summary plots
g. Visualization & Output Generation
• Static plots (PNG)
• Interactive HTML map (Folium) with multiple layers (LST, NDVI, NDBI, land cover, elevation, clusters, hotspots)
• All outputs saved under outputs/ directory
Architectural Patterns & Design Principles
• Modular pipeline: each stage (ingest, preprocess, analysis, modeling, viz) is clearly separated within the script.
• Config-Driven: parameters abstracted into a config module.
• Data-Flow Architecture: linear flow from raw satellite data → transformed features → models → visual artifacts.
• File-System I/O: no long-term datastore, outputs written to disk as artifacts.
• Components: Config Module, GEE Ingestion, Nominatim Helper (optional), Preprocessing, UHI Analysis, ML Training/Prediction, SHAP Interpretability, Visualization Engine.
• External Services: Google Earth Engine, Nominatim API.
• Outputs: Plots directory (PNG), Interactive HTML map.