How ifran-rahman/vcp-ir works
1. Repository purpose and boundary
- This repo centers on VCP-IR, a PyTorch image restoration model for all-in-one restoration tasks such as deblurring, denoising, and deraining.
- The core boundary is the vcp_ir/ package; the top-level scripts (train.py, evaluate.py, sanity_check.py) are orchestration entry points around that package.
2. Core model subsystem
- vcp_ir/model.py contains the main VCP_IR_CBAM architecture.
- The model is described as a Restormer-style hierarchical U-Net with two key additions:
- Visual Conditioning Prompts (VCP): prompts derived from the degraded input itself.
- CBAM attention: channel + spatial attention used inside the network.
3. Model data path
- The input is a degraded image.
- The model first derives a visual embedding from that input and uses it to condition decoder stages.
- The output is a restored image prediction.
- This creates a single forward path: degraded image → conditioned encoder/decoder features → restored image.
4. Prompting and attention mechanism
- The VCP block acts as the main adaptive control mechanism: it converts visual information from the input into a conditioning signal that is injected into decoder stages.
- CBAM refines intermediate features by emphasizing useful channels and spatial regions.
- Architecturally, this means the network is not just a plain U-Net; decoder behavior is dynamically modulated by the input content.
5. Dataset and preprocessing boundary
- vcp_ir/dataset.py provides the restoration dataset logic, named BlurSharpDataset in the README.
- Data is expected in two parallel folders:
- degraded/ for input images
- gt/ for ground-truth clean images
- Files are paired by matching stem names, so the dataset layer is responsible for image pairing, resizing, and batching-ready sample construction.
6. Metrics and evaluation logic
- The dataset/utility layer also exposes PSNR and SSIM metrics.
- These are used in evaluation to quantify restoration quality against the ground truth.
- The evaluation script likely computes these metrics and also writes visual outputs for qualitative inspection.
7. Loss subsystem
- vcp_ir/losses.py contains the training losses.
- The README identifies the primary objective as:
- Charbonnier pixel loss
- plus 0.2 × Laplacian high-frequency L1 loss
- This means training optimizes both low-level pixel fidelity and edge/detail preservation.
8. Training workflow
- train.py is the main experiment runner for model optimization.
- It takes degraded/GT folders, image size, batch size, learning rate, patience, and checkpoint resume path.
- The training boundary is: dataset loader → model forward pass → combined loss → optimizer/early stopping → checkpoint output.
9. Evaluation and visualization workflow
- evaluate.py loads a trained checkpoint and runs inference over the paired dataset.
- Outputs are saved as side-by-side comparison grids: input | prediction | ground truth.
- This script is the main inference/reporting boundary for assessing model quality after training.
10. Sanity and inspection utility
- sanity_check.py provides a quick forward-pass/model-summary style validation.
- It is intended to verify the model shape compatibility and basic runtime behavior before full training.
11. Downstream task analysis area
- The Downstream_Task_Analysis/ folder contains separate subprojects for:
- Aerial_Image_Classificaiton
- Cloud_Cover_Estimation
- These look like downstream evaluation or adaptation experiments rather than part of the core restoration pipeline.
- Architecturally, they form a secondary branch of the repo focused on task-specific analysis around the learned representations.
12. Technology stack and deployment shape
- The repo is built around PyTorch with image-folder based data loading and checkpoint-based training.
- There is no sign of a service backend or distributed system; the architecture is a local research/training pipeline with standalone scripts and saved artifacts (.pth checkpoints, evaluation images).
Main entry points: train.py Train · evaluate.py Evaluate · model.py Model · Readme.md Aerial classification
Components 20
Core restoration
Primary VCP-IR training and inference pipeline.
- Train · Main training runner for restoration experiments.
train.py - Evaluate · Loads checkpoints and runs restoration inference plus reporting.
evaluate.py - Sanity check · Quick forward-pass and shape validation utility.
sanity_check.py - vcp_ir · Core package boundary for model, data, and loss logic.
vcp_ir/__init__.py - Dataset · Pairs degraded and ground-truth images by stem name and prepares batches.
vcp_ir/dataset.py - Model · Defines the VCP_IR_CBAM restoration network.
vcp_ir/model.py - Losses · Training losses combining pixel fidelity and high-frequency detail preservation.
vcp_ir/losses.py - Paired images · Degraded input images matched with clean ground truth targets.
- VCP-IR CBAM · Hierarchical U-Net-style restoration model with visual conditioning prompts and CBAM.
- Visual prompts · Input-derived prompts that modulate decoder stages.
- CBAM · Channel and spatial attention used to refine features.
- PSNR/SSIM · Restoration quality metrics used during evaluation.
Downstream analysis
Task-specific experiments built around the learned representations.
- Aerial classification · Separate downstream task branch for aerial image classification.
Downstream_Task_Analysis/Aerial_Image_Classificaiton/Readme.md - Class train · Training entry point for aerial classification experiments.
Downstream_Task_Analysis/Aerial_Image_Classificaiton/train.py - Class model · Classification model definitions for the aerial task.
Downstream_Task_Analysis/Aerial_Image_Classificaiton/models.py - Class test · Testing or validation entry point for aerial classification.
Downstream_Task_Analysis/Aerial_Image_Classificaiton/test.py - Cloud estimation · Separate downstream task branch for cloud cover estimation.
Downstream_Task_Analysis/Cloud_Cover_Estimation/Readme.md - Cloud eval · Evaluation entry point for cloud cover estimation.
Downstream_Task_Analysis/Cloud_Cover_Estimation/evaluate.py - Cloud metrics · Task-specific metrics for cloud cover estimation.
Downstream_Task_Analysis/Cloud_Cover_Estimation/src/cloud_metrics.py
Other components
- VCP-IR · PyTorch image restoration research repo centered on the VCP-IR model.
Connections 21
- VCP-IR → vcp_ir: core package · The repository is organized around the vcp_ir package.
- VCP-IR → Train: runs · Primary training entry point.
- VCP-IR → Evaluate: runs · Primary inference and reporting entry point.
- VCP-IR → Sanity check: runs · Quick model validation entry point.
- Train → Dataset: loads · Training consumes paired restoration data.
- Train → Model: optimizes · Training forwards data through the restoration model.
- Train → Losses: minimizes · Training uses the composite restoration loss.
- Evaluate → Dataset: loads · Evaluation uses the same paired dataset structure.
- Evaluate → Model: inference · Evaluation restores images with a trained checkpoint.
- Evaluate → PSNR/SSIM: scores · Evaluation reports PSNR and SSIM.
- Sanity check → Model: checks · Sanity script validates model construction and shapes.
- Dataset → Paired images: pairs · Dataset pairs degraded and clean images by matching stems.
- Model → VCP-IR CBAM: defines · Implements the main VCP-IR CBAM architecture.
- VCP-IR CBAM → Visual prompts: conditions on · Decoder stages are modulated by input-derived visual prompts.
- VCP-IR CBAM → CBAM: refines with · Intermediate features are enhanced by channel/spatial attention.
- VCP-IR CBAM → PSNR/SSIM: evaluated by · Restoration output is judged using quality metrics.
- Aerial classification → Class train: contains · Aerial analysis project includes its own training entry.
- Aerial classification → Class model: contains · Classification models live in the aerial project.
- Aerial classification → Class test: contains · Aerial project includes a test/validation script.
- Cloud estimation → Cloud eval: contains · Cloud cover project centers on evaluation.
- Cloud estimation → Cloud metrics: contains · Cloud project provides task-specific metrics.