How ree1s/dit-sr works
Project Type and Purpose
• This is a research‐oriented machine-learning toolkit for real-world image super-resolution and blind face restoration. It implements a diffusion transformer (DiT-SR) architecture, integrates BasicSR utilities, and provides end-to-end pipelines for data ingestion, training, checkpointing, inference, and evaluation.
File Structure Analysis
• Top-level directories:
• configs – YAML configs for different tasks (realsr_DiT, faceir_DiT).
• basicsr – data utilities, datasets, degradation modules, common image utilities inherited from BasicSR.
• datapipe – custom augmentation and degradation pipelines, dataset definitions.
• ldm – latent diffusion model components: autoencoder, transformer/diffusion modules, quantizers, EMA.
• models – core model definitions (UNet, Swin transformer, diffusion solvers, loss functions).
• utils – shared helper functions: I/O, distributed training, logging, option parsing.
• main.py & trainer.py – entry points for training; evaluate.py & inference.py – entry points for evaluation/inference; sampler.py – sampling/denoising logic.
README Insights
• Two training modes (real-world SR, face restoration) invoked via torchrun and distinct config files.
• External checkpoints (autoencoder, loss models) loaded from weights directory.
• Training orchestrated over multi-GPU with PyTorch’s distributed launcher.
• Data lists precomputed in data_list/*.txt and consumed by the data pipeline.
a. Main Components
1. Data Sources
• Raw image datasets (LSDIR, DIV2K, FFHQ) and their path lists in data_list/*.txt.
2. Data Ingestion & Augmentation
• datapipe (aug.py, degradation_bsrgan) + basicsr/data (transforms, samplers, dataloader).
3. Configuration & Launch
• configs/*.yaml loaded by utils/options; torchrun entry in main.py.
4. Model Core
• ldm.autoencoder: encodes images to latents.
• ldm.modules.diffusionmodules + models/unet & swin_transformer: diffusion transformer network.
• ldm.modules.quantize & EMA module.
5. Training Orchestrator
• trainer.py: training loop, loss computation (models/losses), optimizer, scheduler, checkpointing.
• utils/dist_util: multi-GPU sync, logging via utils/logger.
6. Sampling & Inference
• sampler.py + inference.py: denoising loop, checkpoint loading, image reconstruction via autoencoder decoder.
7. Evaluation
• evaluate.py & test scripts (test_realsr.sh, test_faceir.sh): automated benchmarks on real/synthetic datasets.
8. External Dependencies
• PyTorch, CUDA, mmcv, einops, etc. (requirements.txt).
b. Relationships & Data Flow
• Start: User runs torchrun → main.py loads config → instantiate DataModule (datapipe + basicsr/data) → create PyTorch DataLoader → batch of images → pass through autoencoder encoder to produce latents → pass latents + timestep embeddings to diffusion transformer → compute output, decode latents → compute loss via loss modules → optimizer step → periodically save checkpoints → at inference, sampler.py loads checkpoint → reverse diffusion process → decode final latents → save super-resolved images.
c. Architectural Patterns
• Pipeline pattern for data preprocessing and augmentation.
• Modular registry for model components (basicsr.utils.registry).
• Config-driven instantiation (Hydra-style YAML).
• Distributed data-parallel training via torch.distributed.
• Separation of concerns: data, model, training, inference, evaluation.
d. Key Technologies & Frameworks
• PyTorch for tensor operations and model definition.
• BasicSR utilities for standard data handling.
• Custom datapipe for degradations (BSRGAN-style).
• Latent diffusion model architecture in ldm.
• Transformer backbones (UNet, Swin).
• YAML config paradigm, torchrun for multi-GPU.