How arlo0o/uniscene-unified-occupancy-centric-driving-scene-generation works
UniScene generates driving-scene data in stages: BEV layouts condition semantic occupancy generation, then occupancy conditions LiDAR and multi-view video generation. The repository also includes NuScenes occupancy-label preparation and a temporal world-model branch. The README documents the modality handoffs; sampled source confirms dataset loading and parts of occupancy preparation and video rendering, but not every model-to-inference call, so uncertain internal links are omitted.
Main entry points: train_OccDiT.py Occupancy training · dataload_util.py Occupancy datasets · infer.py LiDAR inference · save_bevlayout_12hz.py BEV preparation
Components 19
Data preparation
- BEV preparation
occupancy_gen/12hz_processing/save_bevlayout_12hz.py - GT occupancy prep
data_process/generate_occ.py
Occupancy generation
- Occupancy datasets
occupancy_gen/dataload_util.py - Occupancy training
occupancy_gen/train_OccDiT.py - Occupancy VAE
occupancy_gen/model_vae/VAE/vae_2d_resnet.py - Occupancy diffusion
occupancy_gen/diffusion/gaussian_diffusion.py
LiDAR generation
- LiDAR inference
lidar_gen/tools/infer.py - Occ-to-LiDAR detector
lidar_gen/pcdet/models/detectors/occ2lidar_sparseunet.py - LiDAR rendering head
lidar_gen/pcdet/models/dense_heads/occ2lidar_render.py
Video generation
- Video inference
video_gen/inference_video.py - Occupancy-conditioned render
video_gen/gs_render/render_eval_condition_gt.py - Gaussian renderer
video_gen/gs_render/gaussian_renderer/__init__.py
Temporal modeling
- World-model training
occupancy_gen/wm_part/train_worldmodel.py - World-model evaluation
occupancy_gen/wm_part/eval_worldmodel.py
Other components
- BEV layouts
- NuScenes data
- Semantic occupancy
- LiDAR point clouds
- Multi-view video
Connections 19
- NuScenes data → BEV preparation: provides scenes
- BEV preparation → BEV layouts: produces layouts
- NuScenes data → GT occupancy prep: provides LiDAR labels
- GT occupancy prep → Occupancy datasets: writes occupancy
- BEV layouts → Occupancy training: conditions generation
- Occupancy datasets → Occupancy training: supplies training data
- Occupancy training → Occupancy VAE: uses latent model
- Occupancy training → Occupancy diffusion: uses diffusion
- Occupancy training → Semantic occupancy: generates
- Semantic occupancy → LiDAR inference: conditions generation
- LiDAR inference → Occ-to-LiDAR detector: dispatches
- Occ-to-LiDAR detector → LiDAR rendering head: renders points
- LiDAR inference → LiDAR point clouds: saves
- Semantic occupancy → Occupancy-conditioned render: conditions rendering
- Occupancy-conditioned render → Gaussian renderer: calls
- Gaussian renderer → Multi-view video: renders
- Semantic occupancy → Video inference: provides condition data
- Occupancy datasets → World-model training: supplies sequences
- World-model training → World-model evaluation: trained model