How naver/dust3r works
Project type and purpose
• DUSt3R is a research-oriented 3D reconstruction framework (library + CLI + optional web UI).
• It ingests pairs of images, predicts per-pixel depth and confidence, refines a global 3D point cloud with an optimizer, and visualizes results.
• It includes data preprocessing scripts for multiple datasets, a PyTorch-based model, training and inference entrypoints, and Docker support for demo deployment.
• Raw images & metadata → [Preprocessing layer] → Preprocessed dataset files
• Preprocessed files → [Dataset & Dataloader] → Batches of image pairs
• Batches → [Model Core] → Raw 3D predictions + confidences
• Predictions → [Global Alignment] → Refined poses + merged point cloud
• Refined output → [Visualization] → 3D viewer / evaluation metrics
• Training branch:
• Preprocessed files → [Dataset & Dataloader] → Batches + Labels
• Batches + Labels → [Model Core + Loss Engine] → Backprop → Checkpoints
Architectural patterns
• Modular layering: clear separation between data prep, data loading, model, training, inference, and deployment.
• Plugin-style datasets: each dataset implements a common base interface (base_stereo_view_dataset).
• Optimizer abstraction: cloud_opt packages define interchangeable alignment algorithms.
• Entrypoints (CLI scripts) glue modules into executable workflows.
Technologies & frameworks
• PyTorch (model, training, inference)
• Huggingface Hub (checkpoint retrieval)
• NumPy / SciPy (preprocessing, pair lists)
• Gradio (demo UI)
• Docker / Docker-Compose (containerized deployment)
• submodule: CroCo v2 for RoPE kernels (CUDA extensions)