How chaos96/pointnsp works
PointNSP generates 3D point clouds through a two-stage coarse-to-fine workflow: train a multi-scale VQVAE on ShapeNet point clouds, then train a causal transformer to predict its scale-wise code tokens, and combine the trained models for generation. The VQVAE encodes points with a point-voxel network, forms an FPS-based level-of-detail hierarchy, quantizes features, and decodes point clouds. Transformer training can tokenize online or consume pre-tokenized data. The README documents these capabilities; several implementation details, especially generation and fast-tokenization wiring, were not sampled, so those links follow documented entry points only.
Main entry points: vqvae_model.py Multi-Scale VQVAE · transformer.py Causal Transformer · generate.py Point Cloud Generation · shapenet_data_pc.py Point Loader
Components 19
Point Cloud Data
- ShapeNet Data
- Point Loader
datasets/shapenet_data_pc.py - PartNet Loader
datasets/partnet.py
VQVAE Stage
- VQVAE Training
train_vqvae.py - Multi-Scale VQVAE
model/vqvae_model.py - PVCNN Encoder
model/pvcnn/pvcnn.py - Voxel Branch
model/pvcnn/voxelization.py - FPS LoD
model/fps.py - Point Upsampling
model/upsampling.py - VQ Codebook
model/vqvae_model.py
Transformer Stage
- Online Transformer Training
train_transformer.py - Fast Transformer Training
train_transformer_fast.py - Causal Transformer
model/transformer.py - Causal Masking
model/masking.py - Scale Position Encoding
model/positional_encoding.py
Generation
- Point Cloud Generation
generate.py - Point Cloud Export
utils/visualize.py - Generated Samples
Other components
- Operator
Connections 24
- Operator → VQVAE Training: starts training
- Operator → Online Transformer Training: starts training
- Operator → Fast Transformer Training: starts training
- Operator → Point Cloud Generation: requests samples
- ShapeNet Data → Point Loader: loads points
- Point Loader → VQVAE Training: batches points
- PartNet Loader → VQVAE Training: supplies batches
- VQVAE Training → Multi-Scale VQVAE: optimizes
- Multi-Scale VQVAE → PVCNN Encoder: encodes points
- PVCNN Encoder → Voxel Branch: processes voxel branch
- Multi-Scale VQVAE → FPS LoD: builds LoD
- Multi-Scale VQVAE → VQ Codebook: quantizes features
- Multi-Scale VQVAE → Point Upsampling: refines points
- ShapeNet Data → Point Loader: loads points
- Point Loader → Online Transformer Training: batches points
- Online Transformer Training → Multi-Scale VQVAE: tokenizes with
- Online Transformer Training → Causal Transformer: trains tokens
- Fast Transformer Training → Causal Transformer: trains tokens
- Causal Transformer → Causal Masking: applies causal mask
- Causal Transformer → Scale Position Encoding: uses scale positions
- Point Cloud Generation → Multi-Scale VQVAE: decodes tokens
- Point Cloud Generation → Causal Transformer: predicts tokens
- Point Cloud Generation → Point Cloud Export: exports samples
- Point Cloud Export → Generated Samples: writes files