How deepseek-ai/dreamcraft3d works
DreamCraft3D turns a reference image and text prompt into a coherent textured 3D object through coarse geometry, geometry refinement, and texture refinement. The documented workflow preprocesses the image, trains successive scene representations with view-dependent diffusion guidance, then exports a mesh. The README also describes optional personalized diffusion training from generated multi-view images. Source excerpts establish key system, dataset, geometry, renderer, and guidance components; some subsystem links below reflect documented workflow rather than inspected call sites.
Main entry points: dreamcraft3d.py DreamCraft system · nerf_volume_renderer.py Volume renderer · launch.py Training launcher · implicit_sdf.py Scene geometry
Components 21
Inputs and setup
- Reference image
- Image preprocessing
preprocess_image.py - Experiment config
threestudio/utils/config.py - Training launcher
launch.py - Image dataset
threestudio/data/image.py - Camera sampling
threestudio/data/uncond.py
Scene generation
- DreamCraft system
threestudio/systems/dreamcraft3d.py - Training lifecycle
threestudio/systems/base.py - Scene geometry
threestudio/models/geometry/implicit_sdf.py - Neural networks
threestudio/models/networks.py - Volume renderer
threestudio/models/renderers/nerf_volume_renderer.py - Scene materials
threestudio/models/materials
Diffusion guidance
- Prompt processing
threestudio/models/prompt_processors/base.py - Diffusion guidance
threestudio/models/guidance/stable_diffusion_bsd_guidance.py - Zero123 prior
threestudio/models/guidance/zero123_guidance.py - Latent diffusion core
extern/ldm_zero123/models/diffusion/ddpm.py
Export and personalization
- Personalized diffusion
threestudio/scripts/train_dreambooth_lora.py - Mesh exporter
threestudio/models/exporters/mesh_exporter.py - 3D output
- Multi-view generation
threestudio/scripts/img_to_mv.py
Other components
- Creator
Connections 23
- Creator → Reference image: provides
- Reference image → Image preprocessing: preprocesses
- Creator → Training launcher: starts training
- Experiment config → Training launcher: configures
- Image dataset → DreamCraft system: supplies reference
- Camera sampling → DreamCraft system: samples views
- Training launcher → DreamCraft system: trains stages
- DreamCraft system → Training lifecycle: extends
threestudio/systems/dreamcraft3d.py - DreamCraft system → Scene geometry: optimizes scene
- DreamCraft system → Volume renderer: renders views
- DreamCraft system → Diffusion guidance: requests guidance
threestudio/systems/dreamcraft3d.py - DreamCraft system → Zero123 prior: uses 3D prior
- DreamCraft system → Prompt processing: conditions on prompt
- Scene geometry → Neural networks: uses networks
threestudio/models/geometry/implicit_sdf.py - Volume renderer → Scene geometry: queries geometry
threestudio/models/renderers/nerf_volume_renderer.py - Volume renderer → Scene materials: shades samples
threestudio/models/renderers/nerf_volume_renderer.py - Diffusion guidance → Latent diffusion core: uses diffusion
- Zero123 prior → Latent diffusion core: uses diffusion
- DreamCraft system → Mesh exporter: exports mesh
- Mesh exporter → 3D output: writes mesh
- Reference image → Multi-view generation: generates views
- Multi-view generation → Personalized diffusion: trains on renders
- Personalized diffusion → Diffusion guidance: personalizes prior