How plusgood-steven/id-blau works
Project type and purpose
• ID-Blau is a research-grade image-deblurring pipeline that augments sharp images via an “implicit diffusion-based reBLurring” module, then uses existing state-of-the-art deblurring models (MIMO-UNet, Restormer, Stripformer, FFTformer) to train and evaluate performance.
File structure
• Top-level “PrepareCondition” directory: extracts pixel-wise blur condition maps using RAFT.
• Root scripts:
– diffusion_train.py / diffusion_inference.py → trains and runs the ID-Blau diffusion reblurring model (in models/diffusion_*.py).
– dataloader.py, dataset/ → data I/O for both reblurring and deblurring stages.
– eval_realblur.py, eval_GoPro_HIDE.m → evaluation modules.
• Subdirectories per deblurring model (FFTformer, MIMO_UNet, Restormer, Stripformer): each contains training, inference, loss, and model definition files.
• utils/: shared utilities (flow visualization, conditioning, general I/O).
• weights/: pretrained RAFT and ID-Blau checkpoint, plus deblurring model weights.
Incorporate README insights
• External dependencies: GoPro datasets (GOPRO_Large_all, GOPRO_Large), RAFT optical-flow model.
• Two major workflows:
(a) Data preparation & reblurring augmentation (PrepareCondition → diffusion module).
(b) Deblurring model training, inference, and evaluation (per-model scripts + eval scripts).
Main components & interactions
a. Data Storage & Source
– Raw datasets: GOPRO_Large_all (for flow), GOPRO_Large (sharp images), plus optional RealBlur/HIDE test sets.
– Stored under dataset/ and PrepareCondition/demo-frames.
b. Condition Extraction Module (PrepareCondition)
– Input: GOPRO_Large_all frames + RAFT weights
– Output: blur_condition_maps saved under dataset/GOPRO_flow or custom path
c. Reblurring/Diffusion Module
– diffusion_train.py uses models/diffusion_network.py to learn mapping (sharp + condition → blurred)
– diffusion_inference.py with ID_Blau.pth generates augmented blurred dataset (dataset/GOPRO_Large_Reblur)
d. Deblurring Models Suite
– Four parallel model pipelines: FFTformer, MIMO_UNet, Restormer, Stripformer
– Each has its own train scripts (e.g., deblur_train[_pretrained][_realblur].py), inference script (deblur_predict.py), and model definitions (model.py or layers.py)
– Shared loss utilities under each folder or in root losses/ and loss_util.py
e. Evaluation Module
– MATLAB script eval_GoPro_HIDE.m for GoPro/HIDE metrics
– Python script eval_realblur.py for RealBlur_J / RealBlur_R
f. Utilities & Common Code
– dataloader.py, utils/*.py for data loaders, augmentations, flow visualization, configuration
Architectural patterns & design principles
• Modular pipelines: clear separation between data prep (PrepareCondition), augmentation (Diffusion), deblurring (per-model folders), and evaluation
• Shared components: utility functions and data loaders are reused across modules
Essential elements to capture
• Precise component labels (e.g., “PrepareCondition/generate_condition.py,” “diffusion_train.py,” “MIMO_UNet/deblur_predict.py”)
• Data flow direction and dependencies (e.g., diffusion inference requires pretrained ID_Blau.pth)
• Distinguish code modules vs. data vs. external models/services