How nv-tlabs/artifixer works
ArtiFixer prepares reconstructed scenes, then applies an autoregressive diffusion model to improve rendered video frames; it also supports model training and dataset evaluation. The README documents scene preparation, including reconstruction, rendering, metric-scale estimation, and caption embeddings, plus inference on source or novel camera trajectories. Training and evaluation branches reflect the repository layout and sampled entry points. Some subsystem wiring is inferred only where the README explicitly describes it; unsampled internals are not expanded.
Main entry points: prepare_colmap_artifixer_inputs.py COLMAP preparation · run_inference.py Inference runner · pipeline.py Diffusion pipeline · train.py Training entry
Components 25
Scene preparation
- Prepared scene
- COLMAP preparation
data_processing/prepare_colmap_artifixer_inputs.py - Sparse reconstruction
data_processing/sparse_recon/workflow.py - 3DGRUT rendering
data_processing/render_3dgrut_colmap.py - Metric alignment
data_processing/sparse_recon/metric_alignment.py - Caption embeddings
data_processing/captioning/generate_captions.py
Model training
- Training data
model_training/data/dl3dv_base.py - Training entry
model_training/train.py - Trainer loop
model_training/trainers/trainer_base.py - Distillation
model_training/distillation.py
Inference
- Diffusion pipeline
model_training/pipeline/pipeline.py - KV-cache pipeline
model_training/pipeline/kv_cache_pipeline.py - Video transformer
model_training/net/transformer.py - Video latent codec
model_training/pipeline/pipeline_base.py - Inference runner
model_eval/run_inference.py - COLMAP eval data
model_eval/datasets/reconstructed_colmap_eval.py - Corrected frames
Evaluation
- Evaluation datasets
model_eval/datasets - Evaluation metrics
model_eval/metrics_utils.py - DL3DV metrics
model_eval/compute_metrics_dl3dv.py - Nerfbusters metrics
model_eval/compute_metrics_nerfbusters.py
Other components
- Engineer
- COLMAP scene
- MoGe model
- DL3DV scenes
Connections 25
- Engineer → COLMAP preparation: prepares scene
- COLMAP scene → COLMAP preparation: provides inputs
- COLMAP preparation → Sparse reconstruction: trains reconstruction
- COLMAP preparation → 3DGRUT rendering: renders views
- COLMAP preparation → Metric alignment: estimates scale
- COLMAP preparation → Caption embeddings: generates captions
- Metric alignment → MoGe model: estimates depth
- COLMAP preparation → Prepared scene: writes dataset
- DL3DV scenes → Training data: supplies scenes
- Training data → Training entry: loads batches
- Training entry → Trainer loop: runs training
- Trainer loop → Diffusion pipeline: trains model
- Training entry → Distillation: optional distillation
- Engineer → Inference runner: requests correction
- Prepared scene → COLMAP eval data: loads split
- Inference runner → COLMAP eval data: creates dataset
- Inference runner → Diffusion pipeline: runs inference
- Diffusion pipeline → Video transformer: denoises latents
- KV-cache pipeline → Video transformer: denoises chunks
- Diffusion pipeline → Video latent codec: encodes and decodes
- Inference runner → Corrected frames: writes frames
- Evaluation datasets → DL3DV metrics: provides evaluation data
- Evaluation datasets → Nerfbusters metrics: provides evaluation data
- DL3DV metrics → Evaluation metrics: computes scores
- Nerfbusters metrics → Evaluation metrics: computes scores