How cosuleabianca/eea-pm25 works
This repository is a research pipeline for European PM2.5 forecasting, from EEA data acquisition through dataset preparation, multi-horizon model training, evaluation, and publication outputs. The principal workflow is documented in the README; it also describes weather enrichment, multiple forecasting families, statistical comparisons, and interpretability analyses. The graph preserves those capability branches. Some implementations are unsampled, so their relationships are included only where the README documents the workflow; sampled imports support the shared configuration and deep-learning utility links.
Main entry points: results Forecast evaluation · download_pollutants.py Pollutant acquisition · prepare_ml_dataset.py Feature engineering · coverage_only_v6.py Station coverage
Components 23
Data preparation
- Pollutant acquisition
dataset_build/src/download_pollutants.py - PM2.5 site filter
dataset_build/src/filter_pm25_sites.py - Data processing
dataset_build/src/process_data.py - Feature engineering
dataset_build/src/prepare_ml_dataset.py - Station coverage
dataset_build/src/coverage_only_v6.py - Dataset quality filter
dataset_build/src/dataset_full_realistic_v6.py - High-coverage subset
dataset_build/src/dataset_best_station.py
Forecasting
- Classical forecasters
train_models/global - Deep-learning forecasters
train_models/global - Trained models
models
Evaluation and publication
- Forecast evaluation
results - Diebold-Mariano tests
tests/diebold_mariano_tests.py - SHAP importance
tests/shap_feature_importance.py - LSTM occlusion analysis
tests/occlusion_test_dl.py - Model skill
scripts/compute_model_skill.py - Publication outputs
scripts/generate_publication_outputs.py - Tables and figures
tables
Shared utilities
- Pipeline configuration
utils/config_utils.py - Sequence sampling
train_models/dl_sampling_utils.py - Feature configuration
train_models/feature_config.py
Other components
- Researcher
- EEA API
- Open-Meteo
Connections 26
- Researcher → Pollutant acquisition: starts pipeline
README.md - Pollutant acquisition → EEA API: downloads data
dataset_build/src/download_pollutants.py - Pollutant acquisition → PM2.5 site filter: provides pollutant data
README.md - PM2.5 site filter → Data processing: provides filtered data
README.md - Data processing → Feature engineering: provides processed data
README.md - Feature engineering → Open-Meteo: fetches weather
dataset_build/src/prepare_ml_dataset.py - Feature engineering → Station coverage: provides ML dataset
README.md - Station coverage → Dataset quality filter: provides coverage statistics
README.md - Station coverage → High-coverage subset: provides coverage statistics
dataset_build/src/dataset_best_station.py - Dataset quality filter → Classical forecasters: provides training data
README.md - Dataset quality filter → Deep-learning forecasters: provides training data
README.md - Classical forecasters → Trained models: saves models
README.md - Deep-learning forecasters → Trained models: saves models
README.md - Classical forecasters → Forecast evaluation: writes metrics
README.md - Deep-learning forecasters → Forecast evaluation: writes metrics
README.md - Deep-learning forecasters → Sequence sampling: uses sampling
train_models/global/gam_global_dl.py - Diebold-Mariano tests → Forecast evaluation: tests predictions
README.md - SHAP importance → Trained models: analyzes models
README.md - LSTM occlusion analysis → Trained models: analyzes models
README.md - Model skill → Forecast evaluation: computes skill
README.md - Publication outputs → Tables and figures: generates outputs
README.md - Forecast evaluation → Publication outputs: provides results
README.md - Pollutant acquisition → Pipeline configuration: loads settings
dataset_build/src/download_pollutants.py - Data processing → Pipeline configuration: loads settings
dataset_build/src/process_data.py - Feature engineering → Pipeline configuration: loads settings
dataset_build/src/prepare_ml_dataset.py - Station coverage → Pipeline configuration: loads settings
dataset_build/src/coverage_only_v6.py