How prs-eth/marigold works
Marigold adapts pretrained diffusion models for three dense-image tasks: depth, surface normals, and intrinsic image decomposition (IID). The user-facing workflow is image input → task-specific prediction pipeline → task output; pipelines also support ensembling and uncertainty. Separate training, dataset, evaluation, and preprocessing code supports model development and assessment. The README documents the capabilities, while sampled pipeline and trainer excerpts show some implementation details; several script and dataset relationships are therefore left unconnected where source evidence is unavailable.
Main entry points: marigold_depth_pipeline.py Depth pipeline · marigold_normals_pipeline.py Normals pipeline · marigold_iid_pipeline.py IID pipeline · marigold_depth_trainer.py Depth trainer
Components 24
Prediction
- Depth pipeline
marigold/marigold_depth_pipeline.py - Normals pipeline
marigold/marigold_normals_pipeline.py - IID pipeline
marigold/marigold_iid_pipeline.py
Model development
- Training entrypoints
script - Depth trainer
src/trainer/marigold_depth_trainer.py - Depth LCM trainer
src/trainer/marigold_depth_LCM_trainer.py - IID trainer
src/trainer/marigold_iid_trainer.py - Normals trainer
src/trainer/marigold_normals_trainer.py
Data and evaluation
- Task datasets
src/dataset - Mixed sampler
src/dataset/mixed_sampler.py - Dataset preprocessing
script - Task evaluation
script
Shared components
- Prediction ensemble
marigold/util/ensemble.py - Image utilities
marigold/util/image_util.py - Batch sizing
marigold/util/batchsize.py - Training utilities
src/util - Training losses
src/util/loss.py - Task metrics
src/util/metric.py - Loader utilities
src/util/data_loader.py
Other components
- Image analyst
- Input image
- Depth map
- Surface normals
- Intrinsic maps
Connections 33
- Image analyst → Input image: provides
README.md - Input image → Depth pipeline: predicts depth
- Input image → Normals pipeline: predicts normals
- Input image → IID pipeline: decomposes
- Depth pipeline → Depth map: returns depth
marigold/marigold_depth_pipeline.py - Normals pipeline → Surface normals: returns normals
marigold/marigold_normals_pipeline.py - IID pipeline → Intrinsic maps: returns components
marigold/marigold_iid_pipeline.py - Depth pipeline → Prediction ensemble: ensembles predictions
marigold/marigold_depth_pipeline.py - Normals pipeline → Prediction ensemble: ensembles predictions
marigold/marigold_normals_pipeline.py - IID pipeline → Prediction ensemble: ensembles predictions
marigold/marigold_iid_pipeline.py - Depth pipeline → Image utilities: uses image helpers
marigold/marigold_depth_pipeline.py - Normals pipeline → Image utilities: uses image helpers
marigold/marigold_normals_pipeline.py - IID pipeline → Image utilities: uses image helpers
marigold/marigold_iid_pipeline.py - Depth pipeline → Batch sizing: calculates batches
marigold/marigold_depth_pipeline.py - Normals pipeline → Batch sizing: calculates batches
marigold/marigold_normals_pipeline.py - IID pipeline → Batch sizing: calculates batches
marigold/marigold_iid_pipeline.py - Training entrypoints → Depth trainer: starts training
- Training entrypoints → IID trainer: starts training
- Training entrypoints → Normals trainer: starts training
- Training entrypoints → Mixed sampler: uses sampler
script/depth/train.py - Depth trainer → Depth pipeline: trains model
src/trainer/marigold_depth_trainer.py - Depth LCM trainer → Depth trainer: extends trainer
src/trainer/marigold_depth_LCM_trainer.py - Depth trainer → Training losses: uses losses
src/trainer/marigold_depth_trainer.py - Depth trainer → Task metrics: uses metrics
src/trainer/marigold_depth_trainer.py - Depth trainer → Loader utilities: uses loader helpers
src/trainer/marigold_depth_trainer.py - IID trainer → IID pipeline: trains model
src/trainer/marigold_iid_trainer.py - IID trainer → Training losses: uses losses
src/trainer/marigold_iid_trainer.py - IID trainer → Task metrics: uses metrics
src/trainer/marigold_iid_trainer.py - Normals trainer → Normals pipeline: trains model
src/trainer/marigold_normals_trainer.py - Normals trainer → Training losses: uses losses
src/trainer/marigold_normals_trainer.py - Normals trainer → Task metrics: uses metrics
src/trainer/marigold_normals_trainer.py - Dataset preprocessing → Task datasets: prepares task data
script/depth/train.py - Task evaluation → Task metrics: computes scores