# carlosaccardi/psd-downscaling architecture

> Architecture diagram of the GitHub repository carlosaccardi/psd-downscaling, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-11-19). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/carlosaccardi/psd-downscaling
- Repository: https://github.com/carlosaccardi/psd-downscaling

## Overview

##### Project type and purpose
- This is a research‐oriented deep learning codebase for statistical downscaling of meteorological data (“PSD-Downscaling”).
- It implements multiple models (deterministic U-Net regression, probabilistic UNet with CRPS loss, and CorrDiff conditional diffusion), aiming to translate low-resolution ERA5 data to high-resolution CERRA outputs while preserving PSD characteristics.

##### File structure
- Top-level scripts:
- main.py (CLI entry point for training/evaluation)
- metrics.py (evaluation metrics)
- requirements.txt (dependencies)
- src/
- config/ (configuration loading & defaults)
- data/ (dataset classes for ERA5→CERRA tiles)
- models/
– unet.py
– diffusion.py
– losses/ (Fourier & CRPS loss implementations)
- training/ (training loop utilities)
- utils/ (plotting, helper functions)
- yaml_configs/ (model/hyperparameter definitions by experiment)
- saved_models/ (generated at runtime for checkpoints)
- CI: .github/workflows/pre-commit.yml

##### Key insights from README
- Workflow: main.py reads a YAML config → loads datasets → constructs model → runs training or evaluation → logs to W&B → saves checkpoints.
- Branches split CNN-UNet & CorrDiff vs. CRPS U-Net workflows.
- External systems: Weights & Biases for logging; PyTorch Lightning under the hood (implied by checkpoint conventions).

##### Main components
A. Configuration Layer
- YAML configs (yaml_configs/)
- src/config/base_config.py & legacy_compat.py
B. Data Pipeline
- Dataset loader (src/data/dataset.py)
- Preprocessing utilities (src/utils/utils.py)
C. Model Layer
- UNet (src/models/unet.py)
- Diffusion model (src/models/diffusion.py)
- Loss modules (src/models/losses/)
D. Training Engine
- Training loops & schedulers (src/training/)
- Metrics calculation (metrics.py)
E. Checkpoint & Logging
- Checkpoint IO (PyTorch Lightning .ckpt files in saved_models/)
- Logging service (Weights & Biases)
F. CLI & Entry Point
- main.py orchestrates all layers based on config

##### Relationships & interactions
- main.py reads a YAML config → instantiates Config object →
- Config drives Data Pipeline (path & batch settings) → Data Loader produces batches →
- Model Layer consumes batches, returns predictions → Loss modules compute loss →
- Training Engine updates model parameters, computes metrics →
- Checkpoints are written to disk and metrics/logs are pushed to W&B →
- Evaluation follows the same flow with eval flag.

##### Architectural patterns & design principles
- Modular separation of concerns (config, data, models, training, utils).
- Pluggable model definitions via config → supports swapping UNet/CorrDiff/CRPS.
- External configuration driving behavior (YAML‐driven pipelines).
- Clear I/O boundaries for checkpointing and logging.

## Diagram (Mermaid)

```mermaid
flowchart TB
    %% External Data Sources
    subgraph "External Data" 
        direction TB
        ERA5["ERA5 Data"]:::external
        CERRA["CERRA Data"]:::external
    end

    %% CLI & Entry Point
    Main["main.py\n(CLI Entry Point)"]:::cli

    %% Configuration Layer
    subgraph "Configuration Layer"
        direction TB
        YAML["YAML Config Files"]:::config
        BaseConfig["Base Config Loader"]:::config
        Legacy["Legacy Compatibility"]:::config
    end

    %% Data Pipeline
    subgraph "Data Pipeline"
        direction TB
        Dataset["Dataset Loader"]:::data
        Preproc["Preprocessing Utilities"]:::data
    end

    %% Model Layer
    subgraph "Model Layer"
        direction TB
        UNet["UNet Module"]:::model
        Diffusion["CorrDiff Model"]:::model
        Losses["Loss Modules (Fourier & CRPS)"]:::model
    end

    %% Training Engine
    subgraph "Training Engine"
        direction TB
        Decision{Mode?}:::decision
        Trainer["Training Loop & Schedulers"]:::training
        Metrics["Metrics Calculator"]:::training
    end

    %% Storage & Logging
    subgraph "Storage & Logging"
        direction TB
        Checkpoint["Checkpoint Storage\n(saved_models/)"]:::storage
        Logging["Weights & Biases"]:::external
    end

    %% Flow Connections
    Main -->|reads| YAML
    YAML -->|loaded by| BaseConfig
    YAML -->|legacy options| Legacy
    BaseConfig -->|provides config| Dataset
    BaseConfig -->|provides config| Preproc

    ERA5 --> Dataset
    CERRA --> Dataset
    Dataset --> Preproc
    Preproc --> UNet
    Preproc --> Diffusion

    UNet --> Losses
    Diffusion --> Losses
    Losses --> Decision

    Decision -->|Training| Trainer
    Decision -->|Evaluation| Metrics

    Trainer --> Checkpoint
    Trainer --> Logging
    Metrics --> Logging

    %% Click Events
    click Main "https://github.com/carlosaccardi/psd-downscaling/blob/CNN-UNet/main.py"
    click YAML "https://github.com/carlosaccardi/psd-downscaling/tree/CNN-UNet/yaml_configs/"
    click BaseConfig "https://github.com/carlosaccardi/psd-downscaling/blob/CNN-UNet/src/config/base_config.py"
    click Legacy "https://github.com/carlosaccardi/psd-downscaling/blob/CNN-UNet/src/config/legacy_compat.py"
    click Dataset "https://github.com/carlosaccardi/psd-downscaling/blob/CNN-UNet/src/data/dataset.py"
    click Preproc "https://github.com/carlosaccardi/psd-downscaling/blob/CNN-UNet/src/utils/utils.py"
    click UNet "https://github.com/carlosaccardi/psd-downscaling/blob/CNN-UNet/src/models/unet.py"
    click Diffusion "https://github.com/carlosaccardi/psd-downscaling/blob/CNN-UNet/src/models/diffusion.py"
    click Losses "https://github.com/carlosaccardi/psd-downscaling/tree/CNN-UNet/src/models/losses/"
    click Trainer "https://github.com/carlosaccardi/psd-downscaling/tree/CNN-UNet/src/training/"
    click Metrics "https://github.com/carlosaccardi/psd-downscaling/blob/CNN-UNet/metrics.py"

    %% Styles
    classDef config fill:#ADD8E6,stroke:#000,stroke-width:1px
    classDef data fill:#90EE90,stroke:#000,stroke-width:1px
    classDef model fill:#FFD700,stroke:#000,stroke-width:1px
    classDef training fill:#DDA0DD,stroke:#000,stroke-width:1px
    classDef external fill:#FFA500,stroke:#000,stroke-width:1px
    classDef storage fill:#D3D3D3,stroke:#000,stroke-width:1px
    classDef cli fill:#87CEFA,stroke:#000,stroke-width:1px
    classDef decision fill:#FFFFFF,stroke:#000,stroke-width:2px,stroke-dasharray: 5 5
```
