How joshwuuu/i2sb-flood works
Project type and purpose
• FloodDiff is a research-oriented PyTorch codebase for training and sampling from diffusion models to generate or inpaint flood imagery, evaluating flood prediction and restoration performance. It is neither a web app nor a microservices setup but a modular machine-learning pipeline that orchestrates data loading, model definition, training, sampling, and evaluation.
File structure
• Top-level scripts:
• train.py – training entry point
• sample.py – sampling/generation entry point
• compute_metrices.py – standalone metric computation
• paper_vis.py – visualization for paper figures
• Config directory: YAML templates for various diffusion model setups
• Dataset & corruption modules: data.py, dataset/, corruption/ – data loading, augmentation, corruption transforms
• Core model code:
• guided_diffusion/ – diffusion process primitives, UNet, scheduler, losses
• i2sb/ – VQGAN-based latent modeling and taming-transformer dependencies
• i2sb/base/ – shared building blocks (attention, encoders, diffusion modules, losses)
• Utilities: distributed_util.py, logger.py, ckpt_util, evaluation/ (FID, ResNet-based scores), scripts/ for shell helpers
README for additional insights
• Emphasis on training (python train.py …) and sampling (python sample.py …)
• Dependencies: conda environment, Python 3.8, PyTorch 2.1.0
• Data: external Google Drive link for time-series rainfall data and evaluation images
• Inheritance: adapted from BBDM and NVLabs’ I2SB codebases
Main components
a. Command-line interfaces
• Training CLI (train.py)
• Sampling CLI (sample.py)
• Metric CLI (compute_metrices.py)
b. Configuration Manager
• YAML config loader (configs/)
c. Data Pipeline
• Dataset loader (dataset/, data.py)
• Corruption & augmentation modules (corruption/)
• Distributed data sampler (distributed_util.py)
d. Model Core
• Diffusion engine (guided_diffusion/)
• Latent VQGAN modules (i2sb/VQGAN/)
• Shared base modules (i2sb/base/)
e. Training Orchestration
• Runner (runner.py) – sets up optimizer, scheduler, step loops
• Loss computation (loss.py, guided_diffusion/losses.py)
f. Checkpointing & Logging
• Checkpoint utilities (ckpt_util.py)
• Logging (logger.py)
• Visualization outputs (paper_vis.py)
g. Evaluation & Metrics
• FID utility (evaluation/fid_util.py)
• ResNet-based score (evaluation/resnet.py)
h. External Resources
• Dataset storage (local fs or cloud drive)
• Pretrained models (dropbox links)
Relationships and data flow
• Training flow:
CLI → parse args & load configs → initialize distributed environment → load dataset & apply corruption → feed batches into diffusion model → compute loss → backpropagate → log metrics & save checkpoints
• Sampling flow:
CLI → load checkpoint & configs → initialize model in eval mode → optionally load conditioning data → run sampling loop → save generated images
• Metrics flow:
CLI → load checkpoints or results folder → compute FID/LPIPS/etc. → output scores
• All flows reference common utilities and modules in guided_diffusion and i2sb.
Architectural patterns & design principles
• Modular packages: guided_diffusion and i2sb as pluggable sub-projects
• Separation of concerns: CLI, config, data, model, training loop, evaluation each in its own module
• Reuse & extension: base modules in i2sb/base provide building blocks for both diffusion and latent models
• Distributed training support via distributed_util.py