# joshwuuu/i2sb-flood architecture

> Architecture diagram of the GitHub repository joshwuuu/i2sb-flood, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-06-11). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/joshwuuu/i2sb-flood
- Repository: https://github.com/joshwuuu/i2sb-flood

## Overview

##### Project type and purpose
- FloodDiff is a research-oriented PyTorch codebase for training and sampling from diffusion models to generate or inpaint flood imagery, evaluating flood prediction and restoration performance. It is neither a web app nor a microservices setup but a modular machine-learning pipeline that orchestrates data loading, model definition, training, sampling, and evaluation.

##### File structure
- Top-level scripts:
- train.py – training entry point
- sample.py – sampling/generation entry point
- compute_metrices.py – standalone metric computation
- paper_vis.py – visualization for paper figures
- Config directory: YAML templates for various diffusion model setups
- Dataset & corruption modules: data.py, dataset/, corruption/ – data loading, augmentation, corruption transforms
- Core model code:
- guided_diffusion/ – diffusion process primitives, UNet, scheduler, losses
- i2sb/ – VQGAN-based latent modeling and taming-transformer dependencies
- i2sb/base/ – shared building blocks (attention, encoders, diffusion modules, losses)
- Utilities: distributed_util.py, logger.py, ckpt_util, evaluation/ (FID, ResNet-based scores), scripts/ for shell helpers

##### README for additional insights
- Emphasis on training (`python train.py …`) and sampling (`python sample.py …`)
- Dependencies: conda environment, Python 3.8, PyTorch 2.1.0
- Data: external Google Drive link for time-series rainfall data and evaluation images
- Inheritance: adapted from BBDM and NVLabs’ I2SB codebases

##### Main components
a. Command-line interfaces
- Training CLI (train.py)
- Sampling CLI (sample.py)
- Metric CLI (compute_metrices.py)
b. Configuration Manager
- YAML config loader (configs/)
c. Data Pipeline
- Dataset loader (dataset/, data.py)
- Corruption & augmentation modules (corruption/)
- Distributed data sampler (distributed_util.py)
d. Model Core
- Diffusion engine (guided_diffusion/)
- Latent VQGAN modules (i2sb/VQGAN/)
- Shared base modules (i2sb/base/)
e. Training Orchestration
- Runner (runner.py) – sets up optimizer, scheduler, step loops
- Loss computation (loss.py, guided_diffusion/losses.py)
f. Checkpointing & Logging
- Checkpoint utilities (ckpt_util.py)
- Logging (logger.py)
- Visualization outputs (paper_vis.py)
g. Evaluation & Metrics
- FID utility (evaluation/fid_util.py)
- ResNet-based score (evaluation/resnet.py)
h. External Resources
- Dataset storage (local fs or cloud drive)
- Pretrained models (dropbox links)

##### Relationships and data flow
- Training flow:
CLI → parse args & load configs → initialize distributed environment → load dataset & apply corruption → feed batches into diffusion model → compute loss → backpropagate → log metrics & save checkpoints
- Sampling flow:
CLI → load checkpoint & configs → initialize model in eval mode → optionally load conditioning data → run sampling loop → save generated images
- Metrics flow:
CLI → load checkpoints or results folder → compute FID/LPIPS/etc. → output scores
- All flows reference common utilities and modules in guided_diffusion and i2sb.

##### Architectural patterns & design principles
- Modular packages: guided_diffusion and i2sb as pluggable sub-projects
- Separation of concerns: CLI, config, data, model, training loop, evaluation each in its own module
- Reuse & extension: base modules in i2sb/base provide building blocks for both diffusion and latent models
- Distributed training support via distributed_util.py

## Diagram (Mermaid)

```mermaid
flowchart TD
    subgraph "CLI Scripts"
        Train["Training CLI\ntrain.py"]:::orchestration
        Sample["Sampling CLI\nsample.py"]:::orchestration
        MetricCLI["Metrics CLI\ncompute_metrices.py"]:::orchestration
    end

    subgraph "Configuration Manager"
        Config["Configs\nconfigs/"]:::orchestration
    end

    subgraph "Data Pipeline"
        DistUtil["distributed_util.py"]:::data
        DataLoader["data.py"]:::data
        DatasetDir["dataset/"]:::data
        Corrupt["corruption/"]:::data
    end

    subgraph "Model Core"
        Diffusion["guided_diffusion/"]:::model
        VQGAN["i2sb/VQGAN/"]:::model
        BaseMod["i2sb/base/"]:::model
    end

    subgraph "Training Orchestration"
        Runner["runner.py"]:::orchestration
        LossLocal["loss.py"]:::orchestration
        LossGD["guided_diffusion/losses.py"]:::orchestration
    end

    subgraph "Utilities"
        CKPT["ckpt_util.py"]:::util
        LoggerLocal["logger.py"]:::util
        LoggerGD["guided_diffusion/logger.py"]:::util
        Vis["paper_vis.py"]:::util
    end

    subgraph "Evaluation & Metrics"
        FID["fid_util.py"]:::util
        ResNet["resnet.py"]:::util
    end

    subgraph "Shell Helper Scripts"
        Scripts["scripts/"]:::util
    end

    subgraph "External Resources"
        DatasetStore(("Dataset Storage")):::external
        Pretrained(("Pretrained Models")):::external
    end

    %% Training flow
    Train -->|args & config| Config
    Train -->|init dist| DistUtil
    DistUtil --> DataLoader
    DistUtil --> DatasetDir
    DataLoader --> Corrupt
    Corrupt --> Diffusion
    DatasetDir --> Diffusion
    Config --> Diffusion
    Diffusion --> LossGD
    VQGAN --> LossLocal
    BaseMod --> LossLocal
    LossGD --> Runner
    LossLocal --> Runner
    Runner --> CKPT
    Runner --> LoggerLocal
    Runner --> LoggerGD
    Runner --> Diffusion
    Runner --> VQGAN
    Runner --> BaseMod
    Runner --> DatasetStore
    Runner --> Pretrained

    %% Sampling flow
    Sample --> Config
    Sample --> CKPT
    CKPT --> Diffusion
    CKPT --> VQGAN
    Diffusion --> Vis
    VQGAN --> Vis

    %% Metrics flow
    MetricCLI --> CKPT
    MetricCLI --> FID
    MetricCLI --> ResNet

    %% Scripts connections
    Scripts --> Train
    Scripts --> Sample
    Scripts --> MetricCLI

    %% Click Events
    click Train "https://github.com/joshwuuu/i2sb-flood/blob/master/train.py"
    click Sample "https://github.com/joshwuuu/i2sb-flood/blob/master/sample.py"
    click MetricCLI "https://github.com/joshwuuu/i2sb-flood/blob/master/compute_metrices.py"
    click Config "https://github.com/joshwuuu/i2sb-flood/tree/master/configs/"
    click DistUtil "https://github.com/joshwuuu/i2sb-flood/blob/master/distributed_util.py"
    click DataLoader "https://github.com/joshwuuu/i2sb-flood/blob/master/data.py"
    click DatasetDir "https://github.com/joshwuuu/i2sb-flood/tree/master/dataset/"
    click Corrupt "https://github.com/joshwuuu/i2sb-flood/tree/master/corruption/"
    click Diffusion "https://github.com/joshwuuu/i2sb-flood/tree/master/guided_diffusion/"
    click VQGAN "https://github.com/joshwuuu/i2sb-flood/tree/master/i2sb/VQGAN/"
    click BaseMod "https://github.com/joshwuuu/i2sb-flood/tree/master/i2sb/base/"
    click Runner "https://github.com/joshwuuu/i2sb-flood/blob/master/i2sb/runner.py"
    click LossLocal "https://github.com/joshwuuu/i2sb-flood/blob/master/loss.py"
    click LossGD "https://github.com/joshwuuu/i2sb-flood/blob/master/guided_diffusion/losses.py"
    click CKPT "https://github.com/joshwuuu/i2sb-flood/blob/master/i2sb/ckpt_util.py"
    click LoggerLocal "https://github.com/joshwuuu/i2sb-flood/blob/master/logger.py"
    click LoggerGD "https://github.com/joshwuuu/i2sb-flood/blob/master/guided_diffusion/logger.py"
    click Vis "https://github.com/joshwuuu/i2sb-flood/blob/master/paper_vis.py"
    click FID "https://github.com/joshwuuu/i2sb-flood/blob/master/evaluation/fid_util.py"
    click ResNet "https://github.com/joshwuuu/i2sb-flood/blob/master/evaluation/resnet.py"
    click Scripts "https://github.com/joshwuuu/i2sb-flood/tree/master/scripts/"

    %% Styles
    classDef data fill:#ADD8E6,stroke:#000;
    classDef model fill:#90EE90,stroke:#000;
    classDef orchestration fill:#FFA500,stroke:#000;
    classDef util fill:#DDA0DD,stroke:#000;
    classDef external fill:#cccccc,stroke-dasharray: 5 5;
```
