How 3dv-casia/buildinggpt works
BuildingGPT reconstructs building wireframes from point clouds by conditioning an autoregressive token model on a point-cloud latent representation, then detokenizing generated tokens into wireframe geometry. The principal paths are inference and two-stage training: the README documents autoregressive pretraining followed by preference-score-based DPO post-training. Source excerpts confirm the inference/model components and portions of training and dataset handling; DPO wiring and some subsystem internals are documented but not visible in the sampled code.
Main entry points: models.py Token model · infer.py Inference runner · main.py Training runner · modeling_opt.py OPT decoder
Components 18
Inference
- Inference runner
infer.py
Conditioned generation
- Point conditioning
core/models.py - Point encoder
core/transformer/point.py - Token model
core/models.py - OPT decoder
core/transformer/modeling_opt.py - Attention layers
core/transformer/attention.py
Data and geometry
- Wireframe tokenizer
core/utils.py - Wireframe detokenizer
core/provider.py - Building datasets
core/provider.py
Training
- Model and run options
core/options.py - Training runner
main.py - Autoregressive pretraining
main.py - Preference scoring
core - DPO post-training
core
Other components
- Researcher
- Building point cloud
- Reconstructed wireframe
- Model checkpoint
Connections 19
- Researcher → Inference runner: runs inference
- Building point cloud → Inference runner: provides input
- Inference runner → Point conditioning: prepares condition
- Point conditioning → Point encoder: encodes points
- Point conditioning → Token model: supplies latent
- Inference runner → Token model: generates tokens
- Token model → OPT decoder: decodes sequence
- OPT decoder → Attention layers: uses attention
- Token model → Wireframe detokenizer: passes tokens
- Wireframe detokenizer → Reconstructed wireframe: produces geometry
- Wireframe tokenizer → Building datasets: tokenizes examples
- Training runner → Model and run options: reads options
- Training runner → Building datasets: loads batches
- Training runner → Autoregressive pretraining: runs training
- Autoregressive pretraining → Token model: trains model
- Autoregressive pretraining → Model checkpoint: saves weights
- Preference scoring → DPO post-training: scores preference pairs
- DPO post-training → Token model: aligns model
- Model checkpoint → Inference runner: loads weights