# miha5092/air-quality-field-reconstruction architecture

> Architecture diagram of the GitHub repository miha5092/air-quality-field-reconstruction, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-08-11). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/miha5092/air-quality-field-reconstruction
- Repository: https://github.com/miha5092/air-quality-field-reconstruction

## Overview

#### Purpose and execution surface
This is a Python/Linux research pipeline for reconstructing spatiotemporal air-pollution fields from sparse observations. It trains deterministic and generative models on synthetic simulations, then evaluates transfer to real observations. Primary runnable entry points are experiment modules—especially `experiments/real_dataset_evaluation.py`—plus model-specific training/hyperparameter runners in `src/usage/`.

#### Experiment orchestration
Paper scenarios select models, horizons, noise, tuning, and evaluation regimes. Experiment scripts are the top-level control boundary; cluster shell scripts provide batch/HPC launch variants.
- `experiments/real_dataset_evaluation.py`
- `experiments/fine_tuning.py`
- `experiments/timestep.py`
- `cluster/diffusion.sh`

#### Data layer: synthetic fields and real observations
Datasets separate generated/simulation-based training data from real-world sensor data. Voronoi utilities likely support spatial partitioning or sensor-placement representation; dataset utilities centralize preprocessing/shared transforms.
- `src/datasets/voronoi_datasets.py`
- `src/datasets/vitae_dataset.py`
- `src/datasets/real_obs_dataset.py`
- `src/utils/voronoi.py`

#### Reconstruction model families
The repository compares several interchangeable reconstruction backends: convolutional models, convolutional LSTM models for temporal structure, ViT-AE-style models, and diffusion-based generative reconstruction. Model implementation is separated from execution loops.
- `src/models/vcnn.py`
- `src/models/vclstm.py`
- `src/models/vitae.py`
- `src/models/diffusion.py`

#### Training and hyperparameter paths
Usage modules bind datasets, a selected model family, optimization/training logic, and evaluation. Dedicated diffusion training is distinct from deterministic model training, reflecting different objective/sampling behavior. Hyperparameter drivers exist for VCNN and ViTAE.
- `src/usage/vcnn_training.py`
- `src/usage/vitae_training.py`
- `src/usage/diffusion_training.py`
- `src/usage/vcnn_hyper.py`

#### Generative diffusion boundary
Diffusion is architecturally split into the model, training pipeline, and noise schedule. This boundary supports iterative denoising/sampling and the repository’s noise-magnitude experiments, unlike direct deterministic output generation.
- `src/models/diffusion.py`
- `src/utils/training_pipeline_diffusion.py`
- `src/utils/noise_scheduler.py`
- `experiments/noise_magnitude.py`

#### Evaluation, baselines, and outputs
Evaluation utilities provide common scoring/pipeline behavior across models and datasets; kriging is a separate spatial-statistical baseline path. Notebooks and `load_and_eval.py` are analysis/visualization consumers rather than core runtime services.
- `src/utils/evaluation.py`
- `src/utils/evaluation_pipeline.py`
- `src/usage/kriging.py`
- `load_and_eval.py`

#### Runtime and infrastructure
The system runs under Python 3 on Linux with dependencies pinned in `requirements.txt`. It has no indicated web service, database, or cloud dependency. Batch infrastructure is shell-script based under `cluster/`; persistent artifacts appear to be experiment outputs/plots rather than a managed external storage service.
- `requirements.txt`
- `cluster/`
- `config/train_model.yaml`

Main entry points: [`src/utils/evaluation_pipeline.py`](https://github.com/miha5092/air-quality-field-reconstruction/blob/main/src/utils/evaluation_pipeline.py) (Evaluation pipeline), [`src/usage/vcnn_training.py`](https://github.com/miha5092/air-quality-field-reconstruction/blob/main/src/usage/vcnn_training.py) (Deterministic training), [`src/usage/diffusion_training.py`](https://github.com/miha5092/air-quality-field-reconstruction/blob/main/src/usage/diffusion_training.py) (Diffusion training), [`experiments/real_dataset_evaluation.py`](https://github.com/miha5092/air-quality-field-reconstruction/blob/main/experiments/real_dataset_evaluation.py) (Experiment scenarios)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_orchestration["Experiments"]
  node_experiment_scenarios["Experiment scenarios<br/>experiment modules"]
  node_fine_tuning["Fine-tuning experiment<br/>experiment module<br/>[fine_tuning.py]"]
  node_timestep_experiment["Timestep experiment<br/>experiment module<br/>[timestep.py]"]
  node_noise_experiment["Noise-magnitude experiment<br/>experiment module<br/>[noise_magnitude.py]"]
  node_cluster_launch["HPC batch launch<br/>shell launcher<br/>[diffusion.sh]"]
end

subgraph group_data["Data"]
  node_synthetic_data[("Synthetic field data<br/>simulation dataset<br/>[vitae_dataset.py]")]
  node_real_observations[("Real sensor observations<br/>observation dataset")]
  node_voronoi_data["Voronoi representation<br/>spatial dataset"]
  node_voronoi_utils["Voronoi utilities<br/>spatial transforms<br/>[voronoi.py]"]
end

subgraph group_models["Reconstruction models"]
  node_vcnn["VCNN<br/>convolutional reconstructor<br/>[vcnn.py]"]
  node_vclstm["VCLSTM<br/>temporal reconstructor<br/>[vclstm.py]"]
  node_vitae["ViTAE<br/>ViT-AE reconstructor<br/>[vitae.py]"]
  node_diffusion_model["Diffusion model<br/>generative reconstructor<br/>[diffusion.py]"]
end

subgraph group_execution["Training &amp; evaluation"]
  node_deterministic_training["Deterministic training<br/>training runner<br/>[vcnn_training.py]"]
  node_vitae_training["ViTAE training<br/>training runner<br/>[vitae_training.py]"]
  node_vcnn_hyperparameters["VCNN hyperparameter search<br/>tuning runner<br/>[vcnn_hyper.py]"]
  node_diffusion_training["Diffusion training<br/>training runner"]
  node_diffusion_pipeline["Diffusion pipeline<br/>training pipeline"]
  node_noise_scheduler["Noise scheduler<br/>diffusion schedule<br/>[noise_scheduler.py]"]
  node_evaluation_pipeline["Evaluation pipeline<br/>shared evaluation"]
  node_evaluation_metrics["Evaluation metrics<br/>scoring utilities<br/>[evaluation.py]"]
  node_kriging["Kriging baseline<br/>spatial-statistical baseline<br/>[kriging.py]"]
end

node_python_runtime["Python/Linux runtime<br/>[requirements.txt]"]
node_analysis["Result analysis<br/>analysis consumer<br/>[load_and_eval.py]"]

node_cluster_launch -->|"launches"| node_diffusion_training
node_experiment_scenarios -->|"runs"| node_deterministic_training
node_experiment_scenarios -->|"runs"| node_diffusion_training
node_fine_tuning -->|"fine-tunes"| node_deterministic_training
node_timestep_experiment -->|"evaluates horizons"| node_evaluation_pipeline
node_noise_experiment -->|"varies noise"| node_diffusion_training
node_voronoi_utils -->|"supports"| node_voronoi_data
node_synthetic_data -->|"training data"| node_deterministic_training
node_voronoi_data -->|"spatial input"| node_deterministic_training
node_synthetic_data -->|"training data"| node_diffusion_training
node_vcnn -->|"trained by"| node_deterministic_training
node_vclstm -->|"trained by"| node_deterministic_training
node_vitae -->|"trained by"| node_vitae_training
node_vcnn_hyperparameters -->|"tunes"| node_deterministic_training
node_diffusion_model -->|"trained by"| node_diffusion_training
node_diffusion_pipeline -->|"implements loop"| node_diffusion_training
node_noise_scheduler -->|"controls noise steps"| node_diffusion_pipeline
node_deterministic_training -->|"evaluates"| node_evaluation_pipeline
node_vitae_training -->|"evaluates"| node_evaluation_pipeline
node_diffusion_training -->|"evaluates samples"| node_evaluation_pipeline
node_real_observations -->|"transfer evaluation"| node_evaluation_pipeline
node_evaluation_pipeline -->|"computes"| node_evaluation_metrics
node_real_observations -->|"baseline input"| node_kriging
node_kriging -->|"compares through"| node_evaluation_pipeline
node_evaluation_pipeline -->|"produces results"| node_analysis
node_python_runtime -.->|"executes"| node_experiment_scenarios

click node_python_runtime "https://github.com/miha5092/air-quality-field-reconstruction/blob/main/requirements.txt"
click node_experiment_scenarios "https://github.com/miha5092/air-quality-field-reconstruction/blob/main/experiments/real_dataset_evaluation.py"
click node_fine_tuning "https://github.com/miha5092/air-quality-field-reconstruction/blob/main/experiments/fine_tuning.py"
click node_timestep_experiment "https://github.com/miha5092/air-quality-field-reconstruction/blob/main/experiments/timestep.py"
click node_noise_experiment "https://github.com/miha5092/air-quality-field-reconstruction/blob/main/experiments/noise_magnitude.py"
click node_cluster_launch "https://github.com/miha5092/air-quality-field-reconstruction/blob/main/cluster/diffusion.sh"
click node_synthetic_data "https://github.com/miha5092/air-quality-field-reconstruction/blob/main/src/datasets/vitae_dataset.py"
click node_real_observations "https://github.com/miha5092/air-quality-field-reconstruction/blob/main/src/datasets/real_obs_dataset.py"
click node_voronoi_data "https://github.com/miha5092/air-quality-field-reconstruction/blob/main/src/datasets/voronoi_datasets.py"
click node_voronoi_utils "https://github.com/miha5092/air-quality-field-reconstruction/blob/main/src/utils/voronoi.py"
click node_vcnn "https://github.com/miha5092/air-quality-field-reconstruction/blob/main/src/models/vcnn.py"
click node_vclstm "https://github.com/miha5092/air-quality-field-reconstruction/blob/main/src/models/vclstm.py"
click node_vitae "https://github.com/miha5092/air-quality-field-reconstruction/blob/main/src/models/vitae.py"
click node_diffusion_model "https://github.com/miha5092/air-quality-field-reconstruction/blob/main/src/models/diffusion.py"
click node_deterministic_training "https://github.com/miha5092/air-quality-field-reconstruction/blob/main/src/usage/vcnn_training.py"
click node_vitae_training "https://github.com/miha5092/air-quality-field-reconstruction/blob/main/src/usage/vitae_training.py"
click node_vcnn_hyperparameters "https://github.com/miha5092/air-quality-field-reconstruction/blob/main/src/usage/vcnn_hyper.py"
click node_diffusion_training "https://github.com/miha5092/air-quality-field-reconstruction/blob/main/src/usage/diffusion_training.py"
click node_diffusion_pipeline "https://github.com/miha5092/air-quality-field-reconstruction/blob/main/src/utils/training_pipeline_diffusion.py"
click node_noise_scheduler "https://github.com/miha5092/air-quality-field-reconstruction/blob/main/src/utils/noise_scheduler.py"
click node_evaluation_pipeline "https://github.com/miha5092/air-quality-field-reconstruction/blob/main/src/utils/evaluation_pipeline.py"
click node_evaluation_metrics "https://github.com/miha5092/air-quality-field-reconstruction/blob/main/src/utils/evaluation.py"
click node_kriging "https://github.com/miha5092/air-quality-field-reconstruction/blob/main/src/usage/kriging.py"
click node_analysis "https://github.com/miha5092/air-quality-field-reconstruction/blob/main/load_and_eval.py"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_experiment_scenarios,node_fine_tuning,node_timestep_experiment,node_noise_experiment,node_cluster_launch toneBlue
class node_synthetic_data,node_real_observations,node_voronoi_data,node_voronoi_utils toneAmber
class node_vcnn,node_vclstm,node_vitae,node_diffusion_model toneMint
class node_deterministic_training,node_vitae_training,node_vcnn_hyperparameters,node_diffusion_training,node_diffusion_pipeline,node_noise_scheduler,node_evaluation_pipeline,node_evaluation_metrics,node_kriging toneRose
class node_python_runtime,node_analysis toneNeutral
```

## Components

### Experiments

- [Experiment scenarios](https://github.com/miha5092/air-quality-field-reconstruction/blob/main/experiments/real_dataset_evaluation.py): experiment modules · Select model, horizon, noise, tuning, and evaluation regimes. · `experiments/real_dataset_evaluation.py`
- [Fine-tuning experiment](https://github.com/miha5092/air-quality-field-reconstruction/blob/main/experiments/fine_tuning.py): experiment module · `experiments/fine_tuning.py`
- [Timestep experiment](https://github.com/miha5092/air-quality-field-reconstruction/blob/main/experiments/timestep.py): experiment module · `experiments/timestep.py`
- [Noise-magnitude experiment](https://github.com/miha5092/air-quality-field-reconstruction/blob/main/experiments/noise_magnitude.py): experiment module · `experiments/noise_magnitude.py`
- [HPC batch launch](https://github.com/miha5092/air-quality-field-reconstruction/blob/main/cluster/diffusion.sh): shell launcher · `cluster/diffusion.sh`

### Data

- [Synthetic field data](https://github.com/miha5092/air-quality-field-reconstruction/blob/main/src/datasets/vitae_dataset.py): simulation dataset · Training fields generated from simulations. · `src/datasets/vitae_dataset.py`
- [Real sensor observations](https://github.com/miha5092/air-quality-field-reconstruction/blob/main/src/datasets/real_obs_dataset.py): observation dataset · Transfer-evaluation data from real observations. · `src/datasets/real_obs_dataset.py`
- [Voronoi representation](https://github.com/miha5092/air-quality-field-reconstruction/blob/main/src/datasets/voronoi_datasets.py): spatial dataset · Spatial partitioning or sensor-placement representation. · `src/datasets/voronoi_datasets.py`
- [Voronoi utilities](https://github.com/miha5092/air-quality-field-reconstruction/blob/main/src/utils/voronoi.py): spatial transforms · `src/utils/voronoi.py`

### Reconstruction models

- [VCNN](https://github.com/miha5092/air-quality-field-reconstruction/blob/main/src/models/vcnn.py): convolutional reconstructor · `src/models/vcnn.py`
- [VCLSTM](https://github.com/miha5092/air-quality-field-reconstruction/blob/main/src/models/vclstm.py): temporal reconstructor · `src/models/vclstm.py`
- [ViTAE](https://github.com/miha5092/air-quality-field-reconstruction/blob/main/src/models/vitae.py): ViT-AE reconstructor · `src/models/vitae.py`
- [Diffusion model](https://github.com/miha5092/air-quality-field-reconstruction/blob/main/src/models/diffusion.py): generative reconstructor · Iteratively denoises fields rather than directly predicting outputs. · `src/models/diffusion.py`

### Training & evaluation

- [Deterministic training](https://github.com/miha5092/air-quality-field-reconstruction/blob/main/src/usage/vcnn_training.py): training runner · Binds direct reconstruction models, data, optimization, and evaluation. · `src/usage/vcnn_training.py`
- [ViTAE training](https://github.com/miha5092/air-quality-field-reconstruction/blob/main/src/usage/vitae_training.py): training runner · `src/usage/vitae_training.py`
- [VCNN hyperparameter search](https://github.com/miha5092/air-quality-field-reconstruction/blob/main/src/usage/vcnn_hyper.py): tuning runner · `src/usage/vcnn_hyper.py`
- [Diffusion training](https://github.com/miha5092/air-quality-field-reconstruction/blob/main/src/usage/diffusion_training.py): training runner · Uses diffusion-specific objectives and sampling behavior. · `src/usage/diffusion_training.py`
- [Diffusion pipeline](https://github.com/miha5092/air-quality-field-reconstruction/blob/main/src/utils/training_pipeline_diffusion.py): training pipeline · `src/utils/training_pipeline_diffusion.py`
- [Noise scheduler](https://github.com/miha5092/air-quality-field-reconstruction/blob/main/src/utils/noise_scheduler.py): diffusion schedule · `src/utils/noise_scheduler.py`
- [Evaluation pipeline](https://github.com/miha5092/air-quality-field-reconstruction/blob/main/src/utils/evaluation_pipeline.py): shared evaluation · Common scoring and evaluation flow across datasets and models. · `src/utils/evaluation_pipeline.py`
- [Evaluation metrics](https://github.com/miha5092/air-quality-field-reconstruction/blob/main/src/utils/evaluation.py): scoring utilities · `src/utils/evaluation.py`
- [Kriging baseline](https://github.com/miha5092/air-quality-field-reconstruction/blob/main/src/usage/kriging.py): spatial-statistical baseline · `src/usage/kriging.py`

### Other components

- [Python/Linux runtime](https://github.com/miha5092/air-quality-field-reconstruction/blob/main/requirements.txt): runtime · Pinned research environment dependencies. · `requirements.txt`
- [Result analysis](https://github.com/miha5092/air-quality-field-reconstruction/blob/main/load_and_eval.py): analysis consumer · Loads experiment outputs for analysis and visualization. · `load_and_eval.py`

## Connections

- HPC batch launch → Diffusion training: launches
- Experiment scenarios → Deterministic training: runs
- Experiment scenarios → Diffusion training: runs
- Fine-tuning experiment → Deterministic training: fine-tunes
- Timestep experiment → Evaluation pipeline: evaluates horizons
- Noise-magnitude experiment → Diffusion training: varies noise
- Voronoi utilities → Voronoi representation: supports
- Synthetic field data → Deterministic training: training data
- Voronoi representation → Deterministic training: spatial input
- Synthetic field data → Diffusion training: training data
- VCNN → Deterministic training: trained by
- VCLSTM → Deterministic training: trained by
- ViTAE → ViTAE training: trained by
- VCNN hyperparameter search → Deterministic training: tunes
- Diffusion model → Diffusion training: trained by
- Diffusion pipeline → Diffusion training: implements loop
- Noise scheduler → Diffusion pipeline: controls noise steps
- Deterministic training → Evaluation pipeline: evaluates
- ViTAE training → Evaluation pipeline: evaluates
- Diffusion training → Evaluation pipeline: evaluates samples
- Real sensor observations → Evaluation pipeline: transfer evaluation
- Evaluation pipeline → Evaluation metrics: computes
- Real sensor observations → Kriging baseline: baseline input
- Kriging baseline → Evaluation pipeline: compares through
- Evaluation pipeline → Result analysis: produces results
- Python/Linux runtime → Experiment scenarios: executes