# 3dv-casia/buildinggpt architecture

> Architecture diagram of the GitHub repository 3dv-casia/buildinggpt, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-09-29). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/3dv-casia/buildinggpt
- Repository: https://github.com/3dv-casia/buildinggpt

## Overview

BuildingGPT reconstructs building wireframes from point clouds by conditioning an autoregressive token model on a point-cloud latent representation, then detokenizing generated tokens into wireframe geometry. The principal paths are inference and two-stage training: the README documents autoregressive pretraining followed by preference-score-based DPO post-training. Source excerpts confirm the inference/model components and portions of training and dataset handling; DPO wiring and some subsystem internals are documented but not visible in the sampled code.

Main entry points: [`core/models.py`](https://github.com/3dv-casia/buildinggpt/blob/main/core/models.py) (Token model), [`infer.py`](https://github.com/3dv-casia/buildinggpt/blob/main/infer.py) (Inference runner), [`main.py`](https://github.com/3dv-casia/buildinggpt/blob/main/main.py) (Training runner), [`core/transformer/modeling_opt.py`](https://github.com/3dv-casia/buildinggpt/blob/main/core/transformer/modeling_opt.py) (OPT decoder)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_inference["Inference"]
  node_infer["Inference runner<br/>[infer.py]"]
end

subgraph group_model["Conditioned generation"]
  node_conditioning["Point conditioning<br/>[models.py]"]
  node_pointencoder["Point encoder<br/>[point.py]"]
  node_autoregressive["Token model<br/>[models.py]"]
  node_decoder["OPT decoder<br/>[modeling_opt.py]"]
  node_attention["Attention layers<br/>[attention.py]"]
end

subgraph group_data["Data and geometry"]
  node_tokenizer["Wireframe tokenizer<br/>[utils.py]"]
  node_detokenizer["Wireframe detokenizer<br/>[provider.py]"]
  node_dataset["Building datasets<br/>[provider.py]"]
end

subgraph group_training["Training"]
  node_options["Model and run options<br/>[options.py]"]
  node_trainer["Training runner<br/>[main.py]"]
  node_pretraining["Autoregressive pretraining<br/>[main.py]"]
  node_preference["Preference scoring"]
  node_dpo["DPO post-training"]
end

node_user(("Researcher"))
node_pointcloud["Building point cloud"]
node_wireframe["Reconstructed wireframe"]
node_checkpoint["Model checkpoint"]

node_user -->|"runs inference"| node_infer
node_pointcloud -->|"provides input"| node_infer
node_infer -->|"prepares condition"| node_conditioning
node_conditioning -->|"encodes points"| node_pointencoder
node_conditioning -->|"supplies latent"| node_autoregressive
node_infer -->|"generates tokens"| node_autoregressive
node_autoregressive -->|"decodes sequence"| node_decoder
node_decoder -->|"uses attention"| node_attention
node_autoregressive -->|"passes tokens"| node_detokenizer
node_detokenizer -->|"produces geometry"| node_wireframe
node_tokenizer -->|"tokenizes examples"| node_dataset
node_trainer -->|"reads options"| node_options
node_trainer -->|"loads batches"| node_dataset
node_trainer -->|"runs training"| node_pretraining
node_pretraining -->|"trains model"| node_autoregressive
node_pretraining -->|"saves weights"| node_checkpoint
node_preference -->|"scores preference pairs"| node_dpo
node_dpo -->|"aligns model"| node_autoregressive
node_checkpoint -->|"loads weights"| node_infer

click node_infer "https://github.com/3dv-casia/buildinggpt/blob/main/infer.py"
click node_conditioning "https://github.com/3dv-casia/buildinggpt/blob/main/core/models.py"
click node_pointencoder "https://github.com/3dv-casia/buildinggpt/blob/main/core/transformer/point.py"
click node_autoregressive "https://github.com/3dv-casia/buildinggpt/blob/main/core/models.py"
click node_decoder "https://github.com/3dv-casia/buildinggpt/blob/main/core/transformer/modeling_opt.py"
click node_attention "https://github.com/3dv-casia/buildinggpt/blob/main/core/transformer/attention.py"
click node_tokenizer "https://github.com/3dv-casia/buildinggpt/blob/main/core/utils.py"
click node_detokenizer "https://github.com/3dv-casia/buildinggpt/blob/main/core/provider.py"
click node_dataset "https://github.com/3dv-casia/buildinggpt/blob/main/core/provider.py"
click node_options "https://github.com/3dv-casia/buildinggpt/blob/main/core/options.py"
click node_trainer "https://github.com/3dv-casia/buildinggpt/blob/main/main.py"
click node_pretraining "https://github.com/3dv-casia/buildinggpt/blob/main/main.py"
click node_preference "https://github.com/3dv-casia/buildinggpt/tree/main/core"
click node_dpo "https://github.com/3dv-casia/buildinggpt/tree/main/core"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_infer toneBlue
class node_conditioning,node_pointencoder,node_autoregressive,node_decoder,node_attention toneAmber
class node_tokenizer,node_detokenizer,node_dataset toneMint
class node_options,node_trainer,node_pretraining,node_preference,node_dpo toneRose
class node_user,node_pointcloud,node_wireframe,node_checkpoint toneIndigo
```

## Components

### Inference

- [Inference runner](https://github.com/3dv-casia/buildinggpt/blob/main/infer.py): `infer.py`

### Conditioned generation

- [Point conditioning](https://github.com/3dv-casia/buildinggpt/blob/main/core/models.py): `core/models.py`
- [Point encoder](https://github.com/3dv-casia/buildinggpt/blob/main/core/transformer/point.py): `core/transformer/point.py`
- [Token model](https://github.com/3dv-casia/buildinggpt/blob/main/core/models.py): `core/models.py`
- [OPT decoder](https://github.com/3dv-casia/buildinggpt/blob/main/core/transformer/modeling_opt.py): `core/transformer/modeling_opt.py`
- [Attention layers](https://github.com/3dv-casia/buildinggpt/blob/main/core/transformer/attention.py): `core/transformer/attention.py`

### Data and geometry

- [Wireframe tokenizer](https://github.com/3dv-casia/buildinggpt/blob/main/core/utils.py): `core/utils.py`
- [Wireframe detokenizer](https://github.com/3dv-casia/buildinggpt/blob/main/core/provider.py): `core/provider.py`
- [Building datasets](https://github.com/3dv-casia/buildinggpt/blob/main/core/provider.py): `core/provider.py`

### Training

- [Model and run options](https://github.com/3dv-casia/buildinggpt/blob/main/core/options.py): `core/options.py`
- [Training runner](https://github.com/3dv-casia/buildinggpt/blob/main/main.py): `main.py`
- [Autoregressive pretraining](https://github.com/3dv-casia/buildinggpt/blob/main/main.py): `main.py`
- [Preference scoring](https://github.com/3dv-casia/buildinggpt/tree/main/core): `core`
- [DPO post-training](https://github.com/3dv-casia/buildinggpt/tree/main/core): `core`

### Other components

- **Researcher**
- **Building point cloud**
- **Reconstructed wireframe**
- **Model checkpoint**

## Connections

- Researcher → Inference runner: runs inference
- Building point cloud → Inference runner: provides input
- Inference runner → Point conditioning: prepares condition
- Point conditioning → Point encoder: encodes points
- Point conditioning → Token model: supplies latent
- Inference runner → Token model: generates tokens
- Token model → OPT decoder: decodes sequence
- OPT decoder → Attention layers: uses attention
- Token model → Wireframe detokenizer: passes tokens
- Wireframe detokenizer → Reconstructed wireframe: produces geometry
- Wireframe tokenizer → Building datasets: tokenizes examples
- Training runner → Model and run options: reads options
- Training runner → Building datasets: loads batches
- Training runner → Autoregressive pretraining: runs training
- Autoregressive pretraining → Token model: trains model
- Autoregressive pretraining → Model checkpoint: saves weights
- Preference scoring → DPO post-training: scores preference pairs
- DPO post-training → Token model: aligns model
- Model checkpoint → Inference runner: loads weights