How quan-meng/seen2scene works
Seen2Scene prepares real or synthetic scene data, trains a sparse-grid VAE and flow-matching models, and evaluates generated or completed 3D scenes. The graph preserves the documented export, training, conditioning, and evaluation branches; some links are based on README descriptions because their implementation was not sampled.
Main entry points: train_gen.py Generator trainer · main.py Training entry point · dataset.py Scene dataset · train_vae.py VAE trainer
Components 24
Scene data
- 3D-FRONT export
seen2scene/data/export_front3d.py - ARKitScenes export
seen2scene/data/export_arkitscenes.py - ScanNet++ export
seen2scene/data/export_scannetpp.py - FRONT-3D fusion
seen2scene/data/front3d/integrate.py - Prepared scene files
- Scene dataset
seen2scene/dataset/dataset.py
Learning models
- Sparse VAE
seen2scene/models/fvdb/sparse_vae.py - TSDF scene fields
seen2scene/models/fields.py - Flow matching
seen2scene/models/backbone/flow_matching.py - ControlNet
seen2scene/models/backbone/controlnet.py - Sparse transformer
seen2scene/models/sparse/dit.py - Hierarchical layout
seen2scene/models/layout/get_layout_hierarchical.py
Training and generation
- Training configs
seen2scene/configs/opt.py - Training entry point
seen2scene/main.py - VAE trainer
seen2scene/models/trainer/train_vae.py - Generator trainer
seen2scene/models/trainer/train_gen.py - Trainer base
seen2scene/models/trainer/base.py - Generated scenes
Evaluation
- Evaluation tools
seen2scene/eval/compute_iou_l1_l2_tmd.py - Scene metrics
seen2scene/tools/metrics.py - FID and DINO evaluation
seen2scene/eval/compute_fid_dino.py - Uni3D evaluation
seen2scene/eval/compute_uni3d.py
Other components
- Researcher
- Raw scene data
Connections 32
- Researcher → Training entry point: starts training
- Raw scene data → 3D-FRONT export: exports scenes
- Raw scene data → ARKitScenes export: exports scenes
- Raw scene data → ScanNet++ export: exports scenes
- 3D-FRONT export → FRONT-3D fusion: integrates scans
- 3D-FRONT export → Prepared scene files: writes scenes
- FRONT-3D fusion → Prepared scene files: writes TSDF
- ARKitScenes export → Prepared scene files: writes scenes
- ScanNet++ export → Prepared scene files: writes scenes
- Researcher → Hierarchical layout: requests layout
- Hierarchical layout → Generator trainer: conditions generation
- Training configs → Training entry point: configures run
- Training entry point → Scene dataset: configures splits
- Prepared scene files → Scene dataset: loads scenes
- Scene dataset → VAE trainer: supplies batches
- Scene dataset → Generator trainer: supplies batches
- Training entry point → VAE trainer: dispatches VAE
- Training entry point → Generator trainer: dispatches tasks
- VAE trainer → Trainer base: extends trainer
- Generator trainer → Trainer base: extends trainer
- VAE trainer → Sparse VAE: trains encoder
- Generator trainer → Sparse VAE: encodes latents
- Generator trainer → Flow matching: trains and samples
- Generator trainer → ControlNet: conditions completion
- Flow matching → Sparse transformer: runs backbone
- ControlNet → Sparse transformer: uses backbone
- Sparse VAE → TSDF scene fields: encodes TSDF
- Generator trainer → Generated scenes: writes predictions
- Generated scenes → Evaluation tools: reads meshes
- Evaluation tools → Scene metrics: computes metrics
- Generated scenes → FID and DINO evaluation: evaluates images
- Generated scenes → Uni3D evaluation: evaluates scenes