How iamcreateai/neoverse works
NeoVerse reconstructs a scene from an in-the-wild image or video, lets users design a camera trajectory, renders target-view guidance, and generates a novel-trajectory video. Its runtime combines a 3D reconstructor, camera/geometry utilities, and a Wan video-generation pipeline; the README also documents a Gradio workflow, plug-in reconstructors, and ControlBranch fine-tuning. The excerpts show the CLI’s reconstruction and generation calls and the app’s trajectory-preview flow; pipeline internals and some UI wiring were not sampled, so those links remain at subsystem level.
Main entry points: pipelines NeoVerse pipeline · inference.py CLI inference · app.py Gradio demo · worldmirror.py WorldMirror reconstructor
Components 18
User Workflow
- CLI inference
inference.py - Gradio demo
app.py - Trajectory tools
diffsynth/utils/auxiliary.py - Scene preview
diffsynth/utils/app.py - Generated video
Scene Reconstruction
- WorldMirror reconstructor
diffsynth/auxiliary_models/worldmirror/models/models/worldmirror.py - Depth Anything 3
diffsynth/auxiliary_models/depth_anything_3/api.py - Gaussian scene data
diffsynth/utils/app.py
Video Generation
- NeoVerse pipeline
diffsynth/pipelines - Wan video model
diffsynth/models/wan_video_dit.py - ControlBranch
diffsynth/models/wan_video_neoverse_controller.py - Camera controller
diffsynth/models/wan_video_camera_controller.py - Motion controller
diffsynth/models/wan_video_motion_controller.py - Model manager
diffsynth/models/model_manager.py
Model Training
- Training entry
train.py - Text-image trainer
diffsynth/trainers/text_to_image.py - Training datasets
training/data
Other components
- Image/video user
Connections 17
- Image/video user → CLI inference: provides input
- Image/video user → Gradio demo: uploads media
- CLI inference → Trajectory tools: builds trajectory
- Gradio demo → Trajectory tools: designs trajectory
- CLI inference → NeoVerse pipeline: invokes generation
- Gradio demo → Scene preview: renders preview
- CLI inference → WorldMirror reconstructor: reconstructs views
- WorldMirror reconstructor → Gaussian scene data: returns splats
- Scene preview → Gaussian scene data: visualizes splats
- NeoVerse pipeline → Generated video: returns video
- Model manager → NeoVerse pipeline: loads models
- Training entry → Text-image trainer: starts training
- Text-image trainer → Training datasets: uses data
- Depth Anything 3 → WorldMirror reconstructor: optional backend
- NeoVerse pipeline → ControlBranch: uses control branch
- NeoVerse pipeline → Camera controller: uses camera control
- NeoVerse pipeline → Motion controller: uses motion control