# arlo0o/uniscene-unified-occupancy-centric-driving-scene-generation architecture

> Architecture diagram of the GitHub repository arlo0o/uniscene-unified-occupancy-centric-driving-scene-generation, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-09-29). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/arlo0o/uniscene-unified-occupancy-centric-driving-scene-generation
- Repository: https://github.com/arlo0o/uniscene-unified-occupancy-centric-driving-scene-generation

## Overview

UniScene generates driving-scene data in stages: BEV layouts condition semantic occupancy generation, then occupancy conditions LiDAR and multi-view video generation. The repository also includes NuScenes occupancy-label preparation and a temporal world-model branch. The README documents the modality handoffs; sampled source confirms dataset loading and parts of occupancy preparation and video rendering, but not every model-to-inference call, so uncertain internal links are omitted.

Main entry points: [`occupancy_gen/train_OccDiT.py`](https://github.com/arlo0o/uniscene-unified-occupancy-centric-driving-scene-generation/blob/master/occupancy_gen/train_OccDiT.py) (Occupancy training), [`occupancy_gen/dataload_util.py`](https://github.com/arlo0o/uniscene-unified-occupancy-centric-driving-scene-generation/blob/master/occupancy_gen/dataload_util.py) (Occupancy datasets), [`lidar_gen/tools/infer.py`](https://github.com/arlo0o/uniscene-unified-occupancy-centric-driving-scene-generation/blob/master/lidar_gen/tools/infer.py) (LiDAR inference), [`occupancy_gen/12hz_processing/save_bevlayout_12hz.py`](https://github.com/arlo0o/uniscene-unified-occupancy-centric-driving-scene-generation/blob/master/occupancy_gen/12hz_processing/save_bevlayout_12hz.py) (BEV preparation)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_prep["Data preparation"]
  node_layout_prep["BEV preparation"]
  node_occ_labels["GT occupancy prep<br/>[generate_occ.py]"]
end

subgraph group_occ["Occupancy generation"]
  node_occ_data["Occupancy datasets<br/>[dataload_util.py]"]
  node_occ_train["Occupancy training<br/>[train_OccDiT.py]"]
  node_occ_vae["Occupancy VAE<br/>[vae_2d_resnet.py]"]
  node_occ_diffusion["Occupancy diffusion"]
end

subgraph group_lidar["LiDAR generation"]
  node_lidar_infer["LiDAR inference<br/>[infer.py]"]
  node_lidar_detector["Occ-to-LiDAR detector"]
  node_lidar_render["LiDAR rendering head"]
end

subgraph group_video["Video generation"]
  node_video_infer["Video inference<br/>[inference_video.py]"]
  node_video_condition["Occupancy-conditioned render"]
  node_video_renderer["Gaussian renderer<br/>[__init__.py]"]
end

subgraph group_world["Temporal modeling"]
  node_wm_train["World-model training"]
  node_wm_eval["World-model evaluation<br/>[eval_worldmodel.py]"]
end

node_bev_input["BEV layouts"]
node_nusc[("NuScenes data")]
node_occ_result["Semantic occupancy"]
node_lidar_result["LiDAR point clouds"]
node_video_result["Multi-view video"]

node_nusc -->|"provides scenes"| node_layout_prep
node_layout_prep -->|"produces layouts"| node_bev_input
node_nusc -->|"provides LiDAR labels"| node_occ_labels
node_occ_labels -->|"writes occupancy"| node_occ_data
node_bev_input -->|"conditions generation"| node_occ_train
node_occ_data -->|"supplies training data"| node_occ_train
node_occ_train -->|"uses latent model"| node_occ_vae
node_occ_train -->|"uses diffusion"| node_occ_diffusion
node_occ_train -->|"generates"| node_occ_result
node_occ_result -->|"conditions generation"| node_lidar_infer
node_lidar_infer -->|"dispatches"| node_lidar_detector
node_lidar_detector -->|"renders points"| node_lidar_render
node_lidar_infer -->|"saves"| node_lidar_result
node_occ_result -->|"conditions rendering"| node_video_condition
node_video_condition -->|"calls"| node_video_renderer
node_video_renderer -->|"renders"| node_video_result
node_occ_result -.->|"provides condition data"| node_video_infer
node_occ_data -->|"supplies sequences"| node_wm_train
node_wm_train -.->|"trained model"| node_wm_eval

click node_layout_prep "https://github.com/arlo0o/uniscene-unified-occupancy-centric-driving-scene-generation/blob/master/occupancy_gen/12hz_processing/save_bevlayout_12hz.py"
click node_occ_labels "https://github.com/arlo0o/uniscene-unified-occupancy-centric-driving-scene-generation/blob/master/data_process/generate_occ.py"
click node_occ_data "https://github.com/arlo0o/uniscene-unified-occupancy-centric-driving-scene-generation/blob/master/occupancy_gen/dataload_util.py"
click node_occ_train "https://github.com/arlo0o/uniscene-unified-occupancy-centric-driving-scene-generation/blob/master/occupancy_gen/train_OccDiT.py"
click node_occ_vae "https://github.com/arlo0o/uniscene-unified-occupancy-centric-driving-scene-generation/blob/master/occupancy_gen/model_vae/VAE/vae_2d_resnet.py"
click node_occ_diffusion "https://github.com/arlo0o/uniscene-unified-occupancy-centric-driving-scene-generation/blob/master/occupancy_gen/diffusion/gaussian_diffusion.py"
click node_lidar_infer "https://github.com/arlo0o/uniscene-unified-occupancy-centric-driving-scene-generation/blob/master/lidar_gen/tools/infer.py"
click node_lidar_detector "https://github.com/arlo0o/uniscene-unified-occupancy-centric-driving-scene-generation/blob/master/lidar_gen/pcdet/models/detectors/occ2lidar_sparseunet.py"
click node_lidar_render "https://github.com/arlo0o/uniscene-unified-occupancy-centric-driving-scene-generation/blob/master/lidar_gen/pcdet/models/dense_heads/occ2lidar_render.py"
click node_video_infer "https://github.com/arlo0o/uniscene-unified-occupancy-centric-driving-scene-generation/blob/master/video_gen/inference_video.py"
click node_video_condition "https://github.com/arlo0o/uniscene-unified-occupancy-centric-driving-scene-generation/blob/master/video_gen/gs_render/render_eval_condition_gt.py"
click node_video_renderer "https://github.com/arlo0o/uniscene-unified-occupancy-centric-driving-scene-generation/blob/master/video_gen/gs_render/gaussian_renderer/__init__.py"
click node_wm_train "https://github.com/arlo0o/uniscene-unified-occupancy-centric-driving-scene-generation/blob/master/occupancy_gen/wm_part/train_worldmodel.py"
click node_wm_eval "https://github.com/arlo0o/uniscene-unified-occupancy-centric-driving-scene-generation/blob/master/occupancy_gen/wm_part/eval_worldmodel.py"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_layout_prep,node_occ_labels,node_video_result toneBlue
class node_occ_data,node_occ_train,node_occ_vae,node_occ_diffusion,node_nusc toneAmber
class node_lidar_infer,node_lidar_detector,node_lidar_render toneMint
class node_video_infer,node_video_condition,node_video_renderer toneRose
class node_wm_train,node_wm_eval,node_bev_input,node_occ_result,node_lidar_result toneIndigo
```

## Components

### Data preparation

- [BEV preparation](https://github.com/arlo0o/uniscene-unified-occupancy-centric-driving-scene-generation/blob/master/occupancy_gen/12hz_processing/save_bevlayout_12hz.py): `occupancy_gen/12hz_processing/save_bevlayout_12hz.py`
- [GT occupancy prep](https://github.com/arlo0o/uniscene-unified-occupancy-centric-driving-scene-generation/blob/master/data_process/generate_occ.py): `data_process/generate_occ.py`

### Occupancy generation

- [Occupancy datasets](https://github.com/arlo0o/uniscene-unified-occupancy-centric-driving-scene-generation/blob/master/occupancy_gen/dataload_util.py): `occupancy_gen/dataload_util.py`
- [Occupancy training](https://github.com/arlo0o/uniscene-unified-occupancy-centric-driving-scene-generation/blob/master/occupancy_gen/train_OccDiT.py): `occupancy_gen/train_OccDiT.py`
- [Occupancy VAE](https://github.com/arlo0o/uniscene-unified-occupancy-centric-driving-scene-generation/blob/master/occupancy_gen/model_vae/VAE/vae_2d_resnet.py): `occupancy_gen/model_vae/VAE/vae_2d_resnet.py`
- [Occupancy diffusion](https://github.com/arlo0o/uniscene-unified-occupancy-centric-driving-scene-generation/blob/master/occupancy_gen/diffusion/gaussian_diffusion.py): `occupancy_gen/diffusion/gaussian_diffusion.py`

### LiDAR generation

- [LiDAR inference](https://github.com/arlo0o/uniscene-unified-occupancy-centric-driving-scene-generation/blob/master/lidar_gen/tools/infer.py): `lidar_gen/tools/infer.py`
- [Occ-to-LiDAR detector](https://github.com/arlo0o/uniscene-unified-occupancy-centric-driving-scene-generation/blob/master/lidar_gen/pcdet/models/detectors/occ2lidar_sparseunet.py): `lidar_gen/pcdet/models/detectors/occ2lidar_sparseunet.py`
- [LiDAR rendering head](https://github.com/arlo0o/uniscene-unified-occupancy-centric-driving-scene-generation/blob/master/lidar_gen/pcdet/models/dense_heads/occ2lidar_render.py): `lidar_gen/pcdet/models/dense_heads/occ2lidar_render.py`

### Video generation

- [Video inference](https://github.com/arlo0o/uniscene-unified-occupancy-centric-driving-scene-generation/blob/master/video_gen/inference_video.py): `video_gen/inference_video.py`
- [Occupancy-conditioned render](https://github.com/arlo0o/uniscene-unified-occupancy-centric-driving-scene-generation/blob/master/video_gen/gs_render/render_eval_condition_gt.py): `video_gen/gs_render/render_eval_condition_gt.py`
- [Gaussian renderer](https://github.com/arlo0o/uniscene-unified-occupancy-centric-driving-scene-generation/blob/master/video_gen/gs_render/gaussian_renderer/__init__.py): `video_gen/gs_render/gaussian_renderer/__init__.py`

### Temporal modeling

- [World-model training](https://github.com/arlo0o/uniscene-unified-occupancy-centric-driving-scene-generation/blob/master/occupancy_gen/wm_part/train_worldmodel.py): `occupancy_gen/wm_part/train_worldmodel.py`
- [World-model evaluation](https://github.com/arlo0o/uniscene-unified-occupancy-centric-driving-scene-generation/blob/master/occupancy_gen/wm_part/eval_worldmodel.py): `occupancy_gen/wm_part/eval_worldmodel.py`

### Other components

- **BEV layouts**
- **NuScenes data**
- **Semantic occupancy**
- **LiDAR point clouds**
- **Multi-view video**

## Connections

- NuScenes data → BEV preparation: provides scenes
- BEV preparation → BEV layouts: produces layouts
- NuScenes data → GT occupancy prep: provides LiDAR labels
- GT occupancy prep → Occupancy datasets: writes occupancy
- BEV layouts → Occupancy training: conditions generation
- Occupancy datasets → Occupancy training: supplies training data
- Occupancy training → Occupancy VAE: uses latent model
- Occupancy training → Occupancy diffusion: uses diffusion
- Occupancy training → Semantic occupancy: generates
- Semantic occupancy → LiDAR inference: conditions generation
- LiDAR inference → Occ-to-LiDAR detector: dispatches
- Occ-to-LiDAR detector → LiDAR rendering head: renders points
- LiDAR inference → LiDAR point clouds: saves
- Semantic occupancy → Occupancy-conditioned render: conditions rendering
- Occupancy-conditioned render → Gaussian renderer: calls
- Gaussian renderer → Multi-view video: renders
- Semantic occupancy → Video inference: provides condition data
- Occupancy datasets → World-model training: supplies sequences
- World-model training → World-model evaluation: trained model