# cosuleabianca/eea-pm25 architecture

> Architecture diagram of the GitHub repository cosuleabianca/eea-pm25, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-11-16). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/cosuleabianca/eea-pm25
- Repository: https://github.com/cosuleabianca/eea-pm25

## Overview

This repository is a research pipeline for European PM2.5 forecasting, from EEA data acquisition through dataset preparation, multi-horizon model training, evaluation, and publication outputs. The principal workflow is documented in the README; it also describes weather enrichment, multiple forecasting families, statistical comparisons, and interpretability analyses. The graph preserves those capability branches. Some implementations are unsampled, so their relationships are included only where the README documents the workflow; sampled imports support the shared configuration and deep-learning utility links.

Main entry points: [`results`](https://github.com/cosuleabianca/eea-pm25/tree/main/results) (Forecast evaluation), [`dataset_build/src/download_pollutants.py`](https://github.com/cosuleabianca/eea-pm25/blob/main/dataset_build/src/download_pollutants.py) (Pollutant acquisition), [`dataset_build/src/prepare_ml_dataset.py`](https://github.com/cosuleabianca/eea-pm25/blob/main/dataset_build/src/prepare_ml_dataset.py) (Feature engineering), [`dataset_build/src/coverage_only_v6.py`](https://github.com/cosuleabianca/eea-pm25/blob/main/dataset_build/src/coverage_only_v6.py) (Station coverage)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_data["Data preparation"]
  node_download["Pollutant acquisition"]
  node_site_filter["PM2.5 site filter"]
  node_process["Data processing<br/>[process_data.py]"]
  node_feature["Feature engineering"]
  node_coverage["Station coverage"]
  node_quality_filter["Dataset quality filter"]
  node_best_station["High-coverage subset"]
end

subgraph group_forecast["Forecasting"]
  node_classical["Classical forecasters"]
  node_deep_models["Deep-learning forecasters"]
  node_model_artifacts[("Trained models")]
end

subgraph group_analysis["Evaluation and publication"]
  node_evaluation[("Forecast evaluation")]
  node_dm_test["Diebold-Mariano tests"]
  node_shap["SHAP importance"]
  node_occlusion["LSTM occlusion analysis"]
  node_skill["Model skill"]
  node_publication["Publication outputs"]
  node_tables["Tables and figures"]
end

subgraph group_shared["Shared utilities"]
  node_config["Pipeline configuration<br/>[config_utils.py]"]
  node_dl_sampling["Sequence sampling"]
  node_feature_config["Feature configuration<br/>[feature_config.py]"]
end

node_researcher(("Researcher"))
node_eea{{"EEA API"}}
node_open_meteo{{"Open-Meteo"}}

node_researcher -->|"starts pipeline"| node_download
node_download -->|"downloads data"| node_eea
node_download -->|"provides pollutant data"| node_site_filter
node_site_filter -->|"provides filtered data"| node_process
node_process -->|"provides processed data"| node_feature
node_feature -->|"fetches weather"| node_open_meteo
node_feature -->|"provides ML dataset"| node_coverage
node_coverage -->|"provides coverage statistics"| node_quality_filter
node_coverage -->|"provides coverage statistics"| node_best_station
node_quality_filter -->|"provides training data"| node_classical
node_quality_filter -->|"provides training data"| node_deep_models
node_classical -->|"saves models"| node_model_artifacts
node_deep_models -->|"saves models"| node_model_artifacts
node_classical -->|"writes metrics"| node_evaluation
node_deep_models -->|"writes metrics"| node_evaluation
node_deep_models -->|"uses sampling"| node_dl_sampling
node_dm_test -->|"tests predictions"| node_evaluation
node_shap -->|"analyzes models"| node_model_artifacts
node_occlusion -->|"analyzes models"| node_model_artifacts
node_skill -->|"computes skill"| node_evaluation
node_publication -->|"generates outputs"| node_tables
node_evaluation -->|"provides results"| node_publication
node_download -->|"loads settings"| node_config
node_process -->|"loads settings"| node_config
node_feature -->|"loads settings"| node_config
node_coverage -->|"loads settings"| node_config

click node_config "https://github.com/cosuleabianca/eea-pm25/blob/main/utils/config_utils.py"
click node_download "https://github.com/cosuleabianca/eea-pm25/blob/main/dataset_build/src/download_pollutants.py"
click node_site_filter "https://github.com/cosuleabianca/eea-pm25/blob/main/dataset_build/src/filter_pm25_sites.py"
click node_process "https://github.com/cosuleabianca/eea-pm25/blob/main/dataset_build/src/process_data.py"
click node_feature "https://github.com/cosuleabianca/eea-pm25/blob/main/dataset_build/src/prepare_ml_dataset.py"
click node_coverage "https://github.com/cosuleabianca/eea-pm25/blob/main/dataset_build/src/coverage_only_v6.py"
click node_quality_filter "https://github.com/cosuleabianca/eea-pm25/blob/main/dataset_build/src/dataset_full_realistic_v6.py"
click node_best_station "https://github.com/cosuleabianca/eea-pm25/blob/main/dataset_build/src/dataset_best_station.py"
click node_classical "https://github.com/cosuleabianca/eea-pm25/tree/main/train_models/global"
click node_deep_models "https://github.com/cosuleabianca/eea-pm25/tree/main/train_models/global"
click node_dl_sampling "https://github.com/cosuleabianca/eea-pm25/blob/main/train_models/dl_sampling_utils.py"
click node_feature_config "https://github.com/cosuleabianca/eea-pm25/blob/main/train_models/feature_config.py"
click node_model_artifacts "https://github.com/cosuleabianca/eea-pm25/tree/main/models"
click node_evaluation "https://github.com/cosuleabianca/eea-pm25/tree/main/results"
click node_dm_test "https://github.com/cosuleabianca/eea-pm25/blob/main/tests/diebold_mariano_tests.py"
click node_shap "https://github.com/cosuleabianca/eea-pm25/blob/main/tests/shap_feature_importance.py"
click node_occlusion "https://github.com/cosuleabianca/eea-pm25/blob/main/tests/occlusion_test_dl.py"
click node_skill "https://github.com/cosuleabianca/eea-pm25/blob/main/scripts/compute_model_skill.py"
click node_publication "https://github.com/cosuleabianca/eea-pm25/blob/main/scripts/generate_publication_outputs.py"
click node_tables "https://github.com/cosuleabianca/eea-pm25/tree/main/tables"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_download,node_site_filter,node_process,node_feature,node_coverage,node_quality_filter,node_best_station toneBlue
class node_classical,node_deep_models,node_model_artifacts toneAmber
class node_evaluation,node_dm_test,node_shap,node_occlusion,node_skill,node_publication,node_tables,node_eea toneMint
class node_config,node_dl_sampling,node_feature_config toneRose
class node_researcher,node_open_meteo toneIndigo
```

## Components

### Data preparation

- [Pollutant acquisition](https://github.com/cosuleabianca/eea-pm25/blob/main/dataset_build/src/download_pollutants.py): `dataset_build/src/download_pollutants.py`
- [PM2.5 site filter](https://github.com/cosuleabianca/eea-pm25/blob/main/dataset_build/src/filter_pm25_sites.py): `dataset_build/src/filter_pm25_sites.py`
- [Data processing](https://github.com/cosuleabianca/eea-pm25/blob/main/dataset_build/src/process_data.py): `dataset_build/src/process_data.py`
- [Feature engineering](https://github.com/cosuleabianca/eea-pm25/blob/main/dataset_build/src/prepare_ml_dataset.py): `dataset_build/src/prepare_ml_dataset.py`
- [Station coverage](https://github.com/cosuleabianca/eea-pm25/blob/main/dataset_build/src/coverage_only_v6.py): `dataset_build/src/coverage_only_v6.py`
- [Dataset quality filter](https://github.com/cosuleabianca/eea-pm25/blob/main/dataset_build/src/dataset_full_realistic_v6.py): `dataset_build/src/dataset_full_realistic_v6.py`
- [High-coverage subset](https://github.com/cosuleabianca/eea-pm25/blob/main/dataset_build/src/dataset_best_station.py): `dataset_build/src/dataset_best_station.py`

### Forecasting

- [Classical forecasters](https://github.com/cosuleabianca/eea-pm25/tree/main/train_models/global): `train_models/global`
- [Deep-learning forecasters](https://github.com/cosuleabianca/eea-pm25/tree/main/train_models/global): `train_models/global`
- [Trained models](https://github.com/cosuleabianca/eea-pm25/tree/main/models): `models`

### Evaluation and publication

- [Forecast evaluation](https://github.com/cosuleabianca/eea-pm25/tree/main/results): `results`
- [Diebold-Mariano tests](https://github.com/cosuleabianca/eea-pm25/blob/main/tests/diebold_mariano_tests.py): `tests/diebold_mariano_tests.py`
- [SHAP importance](https://github.com/cosuleabianca/eea-pm25/blob/main/tests/shap_feature_importance.py): `tests/shap_feature_importance.py`
- [LSTM occlusion analysis](https://github.com/cosuleabianca/eea-pm25/blob/main/tests/occlusion_test_dl.py): `tests/occlusion_test_dl.py`
- [Model skill](https://github.com/cosuleabianca/eea-pm25/blob/main/scripts/compute_model_skill.py): `scripts/compute_model_skill.py`
- [Publication outputs](https://github.com/cosuleabianca/eea-pm25/blob/main/scripts/generate_publication_outputs.py): `scripts/generate_publication_outputs.py`
- [Tables and figures](https://github.com/cosuleabianca/eea-pm25/tree/main/tables): `tables`

### Shared utilities

- [Pipeline configuration](https://github.com/cosuleabianca/eea-pm25/blob/main/utils/config_utils.py): `utils/config_utils.py`
- [Sequence sampling](https://github.com/cosuleabianca/eea-pm25/blob/main/train_models/dl_sampling_utils.py): `train_models/dl_sampling_utils.py`
- [Feature configuration](https://github.com/cosuleabianca/eea-pm25/blob/main/train_models/feature_config.py): `train_models/feature_config.py`

### Other components

- **Researcher**
- **EEA API**
- **Open-Meteo**

## Connections

- Researcher → Pollutant acquisition: starts pipeline (evidence: `README.md`)
- Pollutant acquisition → EEA API: downloads data (evidence: `dataset_build/src/download_pollutants.py`)
- Pollutant acquisition → PM2.5 site filter: provides pollutant data (evidence: `README.md`)
- PM2.5 site filter → Data processing: provides filtered data (evidence: `README.md`)
- Data processing → Feature engineering: provides processed data (evidence: `README.md`)
- Feature engineering → Open-Meteo: fetches weather (evidence: `dataset_build/src/prepare_ml_dataset.py`)
- Feature engineering → Station coverage: provides ML dataset (evidence: `README.md`)
- Station coverage → Dataset quality filter: provides coverage statistics (evidence: `README.md`)
- Station coverage → High-coverage subset: provides coverage statistics (evidence: `dataset_build/src/dataset_best_station.py`)
- Dataset quality filter → Classical forecasters: provides training data (evidence: `README.md`)
- Dataset quality filter → Deep-learning forecasters: provides training data (evidence: `README.md`)
- Classical forecasters → Trained models: saves models (evidence: `README.md`)
- Deep-learning forecasters → Trained models: saves models (evidence: `README.md`)
- Classical forecasters → Forecast evaluation: writes metrics (evidence: `README.md`)
- Deep-learning forecasters → Forecast evaluation: writes metrics (evidence: `README.md`)
- Deep-learning forecasters → Sequence sampling: uses sampling (evidence: `train_models/global/gam_global_dl.py`)
- Diebold-Mariano tests → Forecast evaluation: tests predictions (evidence: `README.md`)
- SHAP importance → Trained models: analyzes models (evidence: `README.md`)
- LSTM occlusion analysis → Trained models: analyzes models (evidence: `README.md`)
- Model skill → Forecast evaluation: computes skill (evidence: `README.md`)
- Publication outputs → Tables and figures: generates outputs (evidence: `README.md`)
- Forecast evaluation → Publication outputs: provides results (evidence: `README.md`)
- Pollutant acquisition → Pipeline configuration: loads settings (evidence: `dataset_build/src/download_pollutants.py`)
- Data processing → Pipeline configuration: loads settings (evidence: `dataset_build/src/process_data.py`)
- Feature engineering → Pipeline configuration: loads settings (evidence: `dataset_build/src/prepare_ml_dataset.py`)
- Station coverage → Pipeline configuration: loads settings (evidence: `dataset_build/src/coverage_only_v6.py`)
