How hova88/vggt works
VGGT is a feed-forward scene-reconstruction model: a user supplies one or more images, the model aggregates visual tokens, and prediction heads produce camera parameters, depth, point maps, and tracks. Geometry utilities support conversion between depth and 3D coordinates. The repository also includes a finetuning workflow with CO3D and VKITTI datasets, training, loss calculation, and checkpoint handling, plus a documented COLMAP export demo. Edges reflect the README workflow and sampled source; demo and unsampled subsystem wiring is shown only where documented.
Main entry points: vggt.py VGGT model · trainer.py Trainer · composed_dataset.py Dataset composition · demo_colmap.py COLMAP export demo
Components 21
Inference
- Image preprocessing
vggt/utils/load_fn.py - VGGT model
vggt/models/vggt.py - Token aggregation
vggt/models/aggregator.py - Vision transformer
vggt/layers/vision_transformer.py
Prediction heads
- Camera head
vggt/heads/camera_head.py - Depth and point heads
vggt/heads/dpt_head.py - Track head
vggt/heads/track_head.py - Scene predictions
Geometry and tracking
- Geometry utilities
vggt/utils/geometry.py - Pose conversion
vggt/utils/pose_enc.py - Track prediction
vggt/dependency/track_predict.py - COLMAP export demo
demo_colmap.py
Finetuning
- Training launcher
training/launch.py - Trainer
training/trainer.py - Dataset composition
training/data/composed_dataset.py - Co3D dataset
training/data/datasets/co3d.py - Virtual KITTI dataset
training/data/datasets/vkitti.py - Dynamic dataloader
training/data/dynamic_dataloader.py - Training loss
training/loss.py - Checkpoint handling
training/train_utils/checkpoint.py
Other components
- Image user
Connections 23
- Image user → Image preprocessing: provides images
- Image preprocessing → VGGT model: supplies images
- VGGT model → Token aggregation: aggregates inputs
- Token aggregation → Vision transformer: processes tokens
- VGGT model → Camera head: predicts cameras
- VGGT model → Depth and point heads: predicts maps
- VGGT model → Track head: predicts tracks
- Camera head → Pose conversion: decodes pose
- Pose conversion → Scene predictions: returns camera matrices
- Depth and point heads → Scene predictions: returns depth and points
- Track head → Scene predictions: returns tracks
- Geometry utilities → Scene predictions: unprojects depth
- Track prediction → Scene predictions: predicts 3D tracks
- Training launcher → Trainer: starts training
- Trainer → Dataset composition: instantiates data
- Dataset composition → Co3D dataset: selects dataset
- Dataset composition → Virtual KITTI dataset: selects dataset
- Trainer → Dynamic dataloader: loads batches
- Trainer → VGGT model: trains model
- Trainer → Training loss: computes loss
- Trainer → Checkpoint handling: saves state
- COLMAP export demo → VGGT model: runs inference
- COLMAP export demo → Scene predictions: exports reconstruction