# donggeun-yoon/dgdm architecture

> Architecture diagram of the GitHub repository donggeun-yoon/dgdm, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-05-21). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/donggeun-yoon/dgdm
- Repository: https://github.com/donggeun-yoon/dgdm

## Overview

##### Project type and purpose
- This is a research-oriented machine-learning pipeline implementing “Deterministic Guidance Diffusion Model” (DGDM) for probabilistic weather (or video) forecasting.
- It can run on multiple datasets (Moving MNIST, PNW-Typhoon) by swapping configuration files and dataset classes.

##### File structure
- Top-level script:
- main.py – command-line entrypoint for training and testing
- Register.py – central registry for models, datasets, runners (reflects plugin pattern)
- Configuration:
- configs/*.yaml – dataset-specific hyperparameters (diffusion steps, network depth, data paths)
- Data management:
- data/ – raw data storage and download scripts
- datasets/ – PyTorch-style Dataset subclasses for each data type
- Model definitions:
- model/Unet3D.py – 3D U-Net backbone for spatiotemporal encoding/decoding
- model/BrownianBridgeModel.py – diffusion model wrapper incorporating deterministic guidance
- model/predictor.py, utils.py – helper constructs (schedulers, samplers)
- Training & evaluation orchestration:
- runners/BaseRunner.py – generic training loop (forward, backward, logging)
- runners/BBDMRunner.py – specialized runner tying the diffusion model, EMA, loss functions
- runners/EMA.py – exponential moving average module for model weights
- Utilities:
- utils.py – shared helpers (logging, checkpoint I/O, metric computation)

##### Key insights from README
- Architecture image located at resources/architecture.png outlines data flow from raw inputs → diffusion backbone → sample generation
- Usage revolves around invoking main.py with a config YAML, triggering dataset loading, model instantiation, and runner execution

##### Main components
a. Configuration Loader
– Parses YAML, sets hyperparameters, paths, dataset/model/runner types
b. Registry Module (Register.py)
– Maps string identifiers → classes (Dataset, Model, Runner)
c. Data Pipeline
– Raw data on disk → datasets/*.py → batch tensor loader
d. Model Module
– Unet3D encoder/decoder
– BrownianBridgeModel orchestrator (adds noise, applies reverse diffusion with deterministic guidance)
e. Training Runner
– BaseRunner logic: epoch loop, gradient updates
– BBDMRunner specialization: custom loss, sampling steps
– EMA service tracking smoothed weights
f. Checkpointing & Logging
– utils.py for saving model states, metrics, sample outputs
g. Evaluation/Testing
– Reuses runner to generate probabilistic forecasts and compute statistics

##### Relationships & data flow
- Data Pipeline → minibatch tensors → Training Runner → Model forward/backward → EMA → updated weights
- Training Runner → Checkpoint module (periodic) → disk
- Testing Runner (modeled as a branch) → Model sampling → generated forecast outputs

##### Architectural patterns & design principles
- Separation of concerns: config vs. data vs. model vs. runner
- Plugin/registry: easily swap datasets or model variants without editing core code
- Modular pipeline: data ingestion, diffusion process, training orchestration

##### Relevant technologies & frameworks
- Python 3.9
- PyTorch for tensor operations, model definitions, training loops
- NumPy for data handling
- YAML for config files
- Bash scripts for data download

## Diagram (Mermaid)

```mermaid
flowchart TB
    %% Entry Point
    Main["main.py"]:::entry
    click Main "https://github.com/donggeun-yoon/dgdm/blob/main/main.py"

    %% Configuration Loader
    subgraph "Configuration Loader"
        CM["Template-MovingMNIST.yaml"]:::config
        CT["Template-typhoon.yaml"]:::config
    end
    click CM "https://github.com/donggeun-yoon/dgdm/blob/main/configs/Template-MovingMNIST.yaml"
    click CT "https://github.com/donggeun-yoon/dgdm/blob/main/configs/Template-typhoon.yaml"

    %% Registry
    Registry["Register.py"]:::registry
    click Registry "https://github.com/donggeun-yoon/dgdm/blob/main/Register.py"

    %% Data Pipeline
    subgraph "Data Pipeline"
        RawData["Raw Data (disk)"]:::storage
        DMN["data/moving_mnist/"]:::storage
        DMMN["mmnist.py"]:::data
        DTYP["typhoon.py"]:::data
        DCUST["custom.py"]:::data
    end
    click DMN "https://github.com/donggeun-yoon/dgdm/tree/main/data/moving_mnist/"
    click DMMN "https://github.com/donggeun-yoon/dgdm/blob/main/datasets/mmnist.py"
    click DTYP "https://github.com/donggeun-yoon/dgdm/blob/main/datasets/typhoon.py"
    click DCUST "https://github.com/donggeun-yoon/dgdm/blob/main/datasets/custom.py"

    %% Model Module
    subgraph "Model Module"
        UNet["Unet3D.py"]:::model
        BBM["BrownianBridgeModel.py"]:::model
        PRED["predictor.py"]:::model
        MUTIL["utils.py"]:::model
    end
    click UNet "https://github.com/donggeun-yoon/dgdm/blob/main/model/Unet3D.py"
    click BBM "https://github.com/donggeun-yoon/dgdm/blob/main/model/BrownianBridgeModel.py"
    click PRED "https://github.com/donggeun-yoon/dgdm/blob/main/model/predictor.py"
    click MUTIL "https://github.com/donggeun-yoon/dgdm/blob/main/model/utils.py"

    %% Training & Evaluation Runner
    subgraph "Training & Evaluation Runner"
        BaseR["BaseRunner.py"]:::runner
        BBDMR["BBDMRunner.py"]:::runner
        EMAmod["EMA.py"]:::runner
    end
    click BaseR "https://github.com/donggeun-yoon/dgdm/blob/main/runners/BaseRunner.py"
    click BBDMR "https://github.com/donggeun-yoon/dgdm/blob/main/runners/BBDMRunner.py"
    click EMAmod "https://github.com/donggeun-yoon/dgdm/blob/main/runners/EMA.py"

    %% Utilities
    subgraph "Utilities"
        UTIL["utils.py"]:::util
        RUNUTIL["runners/utils.py"]:::util
    end
    click UTIL "https://github.com/donggeun-yoon/dgdm/blob/main/utils.py"
    click RUNUTIL "https://github.com/donggeun-yoon/dgdm/blob/main/runners/utils.py"

    %% Checkpoints & Logs
    Checkpoint["Checkpoints & Logs"]:::storage

    %% Resources
    Resource["architecture.png"]:::resource
    click Resource "https://github.com/donggeun-yoon/dgdm/blob/main/resources/architecture.png"

    %% Flow connections
    Main --> CM
    Main --> CT
    Main --> Registry
    Registry --> DMN
    Registry --> DMMN
    Registry --> DTYP
    Registry --> DCUST
    DMN --> RawData
    RawData --> DMMN
    DMMN --> BaseR
    DCUST --> BaseR
    DTYP --> BaseR
    BaseR --> UNet
    BaseR --> BBM
    BaseR --> PRED
    BaseR --> MUTIL
    BBM --> UNet
    BBM --> MUTIL
    BaseR --> EMAmod
    EMAmod --> Checkpoint
    BaseR --> UTIL
    BaseR --> RUNUTIL
    BaseR --> Checkpoint
    BaseR --> Resource

    %% Classes
    classDef entry fill:#eef,stroke:#446,stroke-width:1px;
    classDef config fill:#e0ffe0,stroke:#2a2,stroke-width:1px;
    classDef registry fill:#e0f7ff,stroke:#08c,stroke-width:1px;
    classDef storage fill:#ffe0e0,stroke:#c22,stroke-width:1px,stroke-dasharray: 5 5;
    classDef data fill:#fff4e6,stroke:#d68,stroke-width:1px;
    classDef model fill:#f0e6ff,stroke:#85a,stroke-width:1px;
    classDef runner fill:#ffe6f0,stroke:#d35,stroke-width:1px;
    classDef util fill:#f5f5f5,stroke:#888,stroke-width:1px;
    classDef resource fill:#e6f7ff,stroke:#47a,stroke-width:1px;
```
