How zzh-tech/estrnn works
Project Type and Purpose
• This is a research‐grade machine‐learning codebase for video deblurring using an Efficient Spatio‐Temporal Recurrent Neural Network (ESTRNN) and associated dataset (BSD, GOPRO, REDS).
• It provides:
• Data loading and preprocessing for synthetic and real‐world deblurring datasets
• Model definitions for multiple RNN‐based architectures
• Training scripts (single‐GPU, data‐parallel, distributed) with configurable hyperparameters
• Inference scripts to apply trained checkpoints to blurry videos
• Utilities for logging, metrics, and parameter management
High‐Level Components (
a. CLI / Orchestration Layer
• main.py (entry point for training & testing)
• inference.py (entry point for standalone video inference)
b. Data Layer
• data/ (dataset classes and utilities)
• External storage: LMDB, raw image folders, Google Drive or local dataset root
c. Model Layer
• model/ (network architectures: ESTRNN.py, STRCNN.py, SRN.py, etc.; attention, arches, model.py)
d. Training Engine
• train/
• dp.py (single‐machine data‐parallel)
• ddp.py (multi‐GPU distributed)
• trainer.py (training loop orchestration)
• loss.py, optimizer.py, metrics.py, utils.py
e. Inference Engine
• inference.py
• Loading checkpoints, reading frames from src, writing results to dst
f. Configuration and Parameters
• para/parameter.py (default hyperparameters, dataset configs)
g. Utilities
• utils/logger.py (logging, tensorboard sync)
Relationships & Data Flow
• CLI → Configuration
• Parse flags (dataset root, model type, learning rate, batch size, trainer_mode)
• Configuration → Data Layer
• Instantiate dataset (BSD, GOPRO, REDS) via data/*.py
• Data Layer → Training Engine
• DataLoader yields batches to trainer
• Training Engine ↔ Model Layer
• Forward pass, compute loss, backprop, optimizer step
• Training Engine → Utilities
• Log losses/metrics to console and tensorboard
• Training Engine → Storage
• Save checkpoints to ./checkpoints/
• Inference Engine → Model Layer
• Load checkpoint, run forward pass on video frames
• Inference Engine → Storage
• Write deblurred frames/video to dst
Key Architectural Patterns & Principles
• Modular separation: data ingestion, model definition, training logic, inference logic, configuration
• Pluggable architectures: arches.py to select network topology at runtime
• Data‐parallel & Distributed training support (dp.py, ddp.py)
• Clear parameterization: central parameter module for experiment reproducibility
Technologies & Frameworks to Annotate
• Python 3.6+
• PyTorch 1.6+ (nn.Module, DataLoader, DistributedDataParallel)
• LMDB (dataset storage)
• OpenCV, scikit‐image (image I/O & processing)
• thop (model complexity profiling)
• tqdm (progress bars)
• tensorboard (visualization)