How vast-ai-research/mira-scene works
Mira-Scene turns a single image into an editable 3D scene through segmentation, depth estimation, canonical-coordinate and voxel prediction, mesh generation, scene assembly, and environment-map generation. Its inference pipeline manages resumable, case-level stage execution; segmentation also supports interactive review. The repository additionally provides Mira-CCM model/training components, UniDataset data utilities, evaluation, and visualization. The README documents the overall stage order; source excerpts expose selected orchestration and model internals, so undocumented implementation links are omitted.
Main entry points: pipeline.py Stage runner · engine.py Segmentation engine · pipeline_ccm_voxel.py CCM voxel pipeline · 5_construct_scene.py Scene assembly
Components 19
Inference orchestration
- Stage runner
infer_scripts/pipeline.py - Case state
infer_scripts/core/manifest.py
Segmentation and review
- Segmentation engine
infer_scripts/segmentation/engine.py - Automatic segmentation
infer_scripts/segmentation/automatic/runner.py - Interactive review
infer_scripts/segmentation/interactive/review.py
3D generation
- Depth estimation
infer_scripts/utils/depth_estimation.py - CCM inference stage
infer_scripts/2_inference_CCM.py - CCM voxel pipeline
Mira-CCM/src/miraccm/pipelines/shape_synthesis/pipeline_ccm_voxel.py - Object mesh generation
infer_scripts/3_inference_mesh.py - Scene assembly
infer_scripts/5_construct_scene.py - Environment map
infer_scripts/6_generate_environment_map.py
Model and data
- CCM voxel transformer
Mira-CCM/src/miraccm/models/transformers/dit_ccm_voxel.py - Scene datasets
UniDataset/src/UniDataset/Scene - Image and mask transforms
UniDataset/src/UniDataset/utils/img_and_mask_transforms.py - Voxel utilities
UniDataset/src/UniDataset/utils/voxel_utils.py - CCM training system
Mira-CCM/src/miraccm/systems/shape_synthesis/system_ccm_voxel.py
Evaluation and viewing
- Scene evaluation
eval_scripts/eval_scene.py - Result visualization
visualization/visualize_results.py
Other components
- Input image
Connections 17
- Input image → Stage runner: submits image
README.md - Stage runner → Segmentation engine: runs segmentation
README.md - Stage runner → Depth estimation: runs depth
README.md - Stage runner → CCM inference stage: dispatches CCM
infer_scripts/pipeline.py - Stage runner → Object mesh generation: runs mesh generation
README.md - Stage runner → Scene assembly: runs assembly
README.md - Stage runner → Environment map: runs map generation
README.md - Stage runner → Case state: tracks stage state
infer_scripts/pipeline.py - Segmentation engine → Automatic segmentation: uses automatic runner
infer_scripts/segmentation/engine.py - Segmentation engine → Interactive review: publishes review inputs
infer_scripts/segmentation/engine.py - CCM inference stage → CCM voxel pipeline: runs inference
infer_scripts/2_inference_CCM.py - CCM voxel pipeline → CCM voxel transformer: uses transformer
Mira-CCM/src/miraccm/pipelines/shape_synthesis/pipeline_ccm_voxel.py - Scene datasets → Image and mask transforms: uses transforms
UniDataset/src/UniDataset/Scene/infinigen_composite_scene_depth.py - Scene datasets → Voxel utilities: uses voxel utilities
UniDataset/src/UniDataset/Scene/infinigen_composite_scene_depth.py - CCM training system → CCM voxel pipeline
- Scene evaluation → Scene assembly: evaluates results
- Result visualization → Scene assembly: visualizes results