How mnthn2326/liss4-cloud-removal works
Cloud Removal GAN is a conditional GAN pipeline that transforms paired cloudy and cloud-free satellite imagery into trained cloud-removal models. The principal workflow is: discover and split raw image pairs, tile and normalize them, load patch manifests through a PyTorch dataset, train a U-Net generator against a PatchGAN discriminator with adversarial and L1 losses, then save checkpoints and log experiments. Evaluation and model registration are documented but currently TODO; DVC and DagsHub provide reproducibility and remote storage/tracking.
Components 18
Data Preparation
- Raw Imagery
- Dataset Config
configs/dataset_rice.yaml - Preprocessing
src/data/preprocess.py - Patch Store
src/data/preprocess.py - Split Manifests
src/data/preprocess.py - Cloud Dataset
src/data/dataset.py
Model Architectures
- U-Net Generator
src/models/generator.py - PatchGAN Discriminator
src/models/discriminator.py
Training Runtime
- GAN Losses
src/training/losses.py - Training Loop
src/training/train.py - Model Checkpoints
src/training/train.py
Evaluation
- Test Evaluation
src/evaluation/evaluate.py - Evaluation Metrics
src/evaluation/evaluate.py
MLOps Services
- Model Registration
src/training/register_model.py - DVC Pipeline
pipelines/dvc.yaml
Other components
- Engineer
- MLflow Tracking
- DagsHub Storage
Connections 21
- Engineer → Preprocessing: runs
- Engineer → Training Loop: runs
- Engineer → Test Evaluation: runs
- Engineer → Model Registration: runs
- Engineer → DVC Pipeline: reproduces
- Preprocessing → Dataset Config: reads
- Preprocessing → Raw Imagery: reads
- Preprocessing → Patch Store: writes
- Preprocessing → Split Manifests: writes
- Cloud Dataset → Split Manifests: reads
- Cloud Dataset → Patch Store: loads
- Training Loop → Cloud Dataset: loads
- Training Loop → U-Net Generator: trains
- Training Loop → PatchGAN Discriminator: trains
- Training Loop → GAN Losses: computes
- Training Loop → Model Checkpoints: saves
- Training Loop → MLflow Tracking: logs
- DVC Pipeline → Raw Imagery: versions
- DVC Pipeline → DagsHub Storage: syncs
- Model Registration → MLflow Tracking: registers
- Test Evaluation → Evaluation Metrics: produces