How hrithik-v/phyre-mamba works
1. What this repo is
- PhyRe-Mamba is a training/inference codebase for underwater image restoration.
- The core pipeline is: UIEB-style dataset → physics feature augmentation → PhyRe-Mamba model → flow-matching training → checkpointed evaluation.
2. Main entry points
- train.py is the primary orchestration script for full training.
- overfit_test.py is a small-sample sanity check to verify the pipeline can learn.
- verify_arch.py checks that the model can be instantiated and run a forward pass.
- test.py is the separate testing/inference entry point.
3. Configuration boundary
- config.yaml is the central control plane for dataset paths, image size, batch size, learning rate, epochs, validation frequency, and checkpoint location.
- src/utils/config.py loads that YAML into the runtime scripts, so the architecture is largely config-driven rather than hardcoded.
4. Data subsystem
- src/data/dataset.py defines the UIEB dataset handling and train/validation split.
- The dataset expects paired input/target underwater images laid out under the configured dataset root.
- This layer is the boundary between raw filesystem data and model-ready tensors.
5. Physics-guided feature generation
- src/data/physics.py adds domain features such as dark channel prior, transmission, and depth.
- These are important because the model is not purely image-to-image; it injects physical underwater priors as additional guidance channels/features.
6. Model architecture
- src/model/phyre_mamba.py is the top-level PhyRe-Mamba network definition.
- src/model/encoder.py provides the CNN encoder.
- src/model/blocks.py contains the Mamba-based state-space block wrapper (VSSBlock).
- src/model/decoder.py implements the decoder, described in the README as a cross-latent decoder.
- Overall, the model combines local convolutional encoding with global sequence/state-space modeling and a reconstruction decoder.
7. Training objective
- src/train/loss.py defines the losses used for learning.
- The README indicates a combination of Flow Matching / rectified flow and perceptual loss.
- This means the training signal is not just pixel regression; it is shaped to support smooth restoration dynamics and perceptual fidelity.
8. Training loop responsibilities
- src/train/trainer.py is the central runtime controller for epoch/batch iteration.
- It handles logging of loss, PSNR, and validation MSE/PSNR, performs validation at val_freq, and manages checkpointing.
- It also saves the best model and periodic checkpoints, so the trainer is both the optimization engine and the experiment manager.
9. Metrics and artifacts
- The repo treats PSNR as the main quality metric during training and validation, with MSE used for validation selection.
- Checkpoints are written to the configured save_dir (default ./checkpoints), with a best-model file and periodic snapshots.
- This creates a clear artifact boundary between ephemeral training state and persisted models.
10. Runtime and infrastructure assumptions
- The project depends on CUDA-capable builds of mamba-ssm and causal-conv1d, so GPU support is material to the architecture.
- Training can use multiple GPUs through PyTorch DataParallel when available.
- The repo is therefore organized around a GPU-backed research training workflow, not a deployment service.
11. How the pieces connect
- In a typical run, a script reads config.yaml, constructs the dataset, augments inputs with physics features, builds the PhyRe-Mamba model, and hands everything to the trainer.
- The trainer computes the flow-based loss, logs metrics, validates periodically, and writes checkpoints.
- The overfit and verify scripts provide lightweight preflight checks before full-scale training.
12. Useful architecture diagram boundaries
- Draw four major boxes: Configuration, Data/Physics preprocessing, Model, and Training/Evaluation.
- Show the data flow from dataset files into physics-guided tensors, then into the encoder/Mamba/decoder stack, then into loss/metrics/checkpoints.
- Treat train.py, overfit_test.py, verify_arch.py, and test.py as orchestration entry points around the same core components.
Main entry points: trainer.py Trainer · phyre_mamba.py PhyRe-Mamba · config.py Config load · blocks.py VSS block
Components 17
Configuration
Central runtime settings and config loading.
- config.yaml · Central control plane for dataset, training, and checkpoint settings.
config.yaml - Config load · Loads YAML settings into the training and inference scripts.
src/utils/config.py
Data and Physics
Dataset loading plus physics-guided feature augmentation.
- UIEB dataset · Reads paired underwater input/target images and handles train/validation split.
src/data/dataset.py - Physics feats · Adds domain priors such as dark channel prior, transmission, and depth.
src/data/physics.py
Model
PhyRe-Mamba network stack.
- PhyRe-Mamba · Top-level underwater image restoration network.
src/model/phyre_mamba.py - Encoder · Convolutional feature extractor for local image structure.
src/model/encoder.py - VSS block · Mamba-based state-space block wrapper for sequence modeling.
src/model/blocks.py - Cross-latent decoder · Decodes latent features back into restored imagery.
src/model/decoder.py
Training and Evaluation
Optimization, metrics, checkpointing, and entry points.
- train.py · Primary orchestration script for full training runs.
train.py - overfit_test.py · Small-sample overfit check for pipeline and learning sanity.
overfit_test.py - verify_arch.py · Instantiates the model and runs a forward-pass check.
verify_arch.py - test.py · Separate testing and inference entry point.
test.py - Trainer · Coordinates epochs, validation, logging, and checkpointing.
src/train/trainer.py - Flow loss · Combines flow-matching / rectified-flow objective with perceptual loss.
src/train/loss.py - PSNR / MSE · Tracks training quality and validation selection signals.
- Checkpoints · Best model and periodic snapshots saved under the configured directory.
- CUDA Mamba · GPU-backed dependencies for mamba-ssm and causal-conv1d, with optional multi-GPU DataParallel.
Connections 18
- config.yaml → Config load: loaded by · YAML settings are parsed into runtime configuration objects.
- Config load → UIEB dataset: dataset paths · Configuration supplies dataset root, split, and image sizing.
- Config load → Trainer: train params · Configuration drives batch size, learning rate, epochs, validation frequency, and save directory.
- Config load → PhyRe-Mamba: model params · Configuration influences model construction and runtime choices.
- UIEB dataset → Physics feats: augmented by · Loaded paired images are enriched with underwater physics features.
- Physics feats → PhyRe-Mamba: feature input · Physics-guided channels are fed into the restoration model.
- PhyRe-Mamba → Encoder: encode · The model begins with convolutional feature extraction.
- Encoder → VSS block: mix · Encoded features are processed by Mamba-based state-space blocks.
- VSS block → Cross-latent decoder: decode · Latent features are reconstructed through the decoder stack.
- Trainer → Flow loss: optimizes · Trainer computes the flow-based training objective.
- Trainer → PSNR / MSE: logs · Trainer tracks PSNR and validation MSE/PSNR.
- Trainer → Checkpoints: writes · Trainer persists best and periodic checkpoints.
- train.py → Trainer: runs · Main training script launches the full training controller.
- overfit_test.py → Trainer: sanity run · Overfit test uses the same core training path on a small sample.
- test.py → PhyRe-Mamba: inference · Testing uses the trained model for restoration inference.
- verify_arch.py → PhyRe-Mamba: forward pass · Architecture verification instantiates and exercises the model.
- CUDA Mamba → VSS block: accelerates · The Mamba blocks rely on CUDA-capable dependencies.
- CUDA Mamba → Trainer: parallelism · Training can use multi-GPU DataParallel when available.