How janspiry/image-super-resolution-via-iterative-refinement works
Project Type and Purpose
• This is a research-oriented ML pipeline for image super-resolution using diffusion models (DDPM/SR3).
• It provides scripts for data preparation, model training (sr.py for conditional SR, sample.py for unconditional sampling), evaluation, and inference.
• It integrates with Weights & Biases for experiment tracking.
Main Architectural Components
a. User Interface / CLI
• Entry points: sr.py, sample.py, eval.py, infer.py, data/prepare_data.py
• Parses command-line args and a JSON config (from config/*.json).
b. Configuration Layer
• JSON files under config/
• Hyperparameters, dataset paths, model architecture, resume state.
c. Data Layer
• Raw dataset directories under dataset/…
• Data-prep script (data/prepare_data.py) generates low-res, high-res, bicubic SR images into dataset/...
• Data loader classes (data/LRHR_dataset.py, data/util.py) wrap LMDB or image folders.
d. Model Layer
• Core diffusion model implementations in model/ddpm_modules/ and model/sr3_modules/
• Building blocks: model/networks.py (UNet), model/base_model.py, model/model.py
• core/ utilities: logger.py, metrics.py, wandb_logger.py
e. Training Engine
• sr.py: orchestrates conditional SR training & validation
• sample.py: unconditional generation
• Uses PyTorch DataLoader → Model → Loss → Optimizer loop
• Hooks into core/metrics.py and optionally core/wandb_logger.py
f. Evaluation Module
• eval.py: loads generated results, computes SSIM/PSNR via core/metrics.py
g. Inference Module
• infer.py: loads a trained model checkpoint and a single LR image → outputs SR image
h. Logging/Monitoring
• Optional Weights & Biases integration via core/wandb_logger.py and CLI flags
• Checkpointing (PyTorch .pt files) and W&B Artifacts
i. External Dependencies
• PyTorch, torchvision, lmdb (optional), wandb
• Pre-trained model weights (downloaded externally)
Relationships & Data Flows
• CLI → reads Config → invokes Data Layer & Model Layer within Training Engine
• Data Layer → feeds mini-batches to Model Layer
• Model Layer → computes outputs → loss → backprop
• Core Logger & Metrics → record training progress → optional W&B API
• Trained checkpoints → stored locally / W&B –> reused by Inference/Evaluation
• Evaluation Module → reads SR results + ground truth → Metrics → report
• Inference Module → reads checkpoint + LR image → Model → outputs SR image
Architectural Patterns & Principles
• Clear separation of concerns: Data vs. Model vs. Training vs. Evaluation vs. Logging
• Config-driven design: everything—from model size to dataset paths—is defined in JSON
• Modular diffusion building blocks (DDPM vs. SR3) under respective subpackages
• Pluggable logging (console vs. W&B) via strategy pattern in core/wandb_logger.py
• Single responsibility for each script: one for training, one for sampling, one for eval, one for infer