# nv-tlabs/artifixer architecture

> Architecture diagram of the GitHub repository nv-tlabs/artifixer, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-09-29). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/nv-tlabs/artifixer
- Repository: https://github.com/nv-tlabs/artifixer

## Overview

ArtiFixer prepares reconstructed scenes, then applies an autoregressive diffusion model to improve rendered video frames; it also supports model training and dataset evaluation. The README documents scene preparation, including reconstruction, rendering, metric-scale estimation, and caption embeddings, plus inference on source or novel camera trajectories. Training and evaluation branches reflect the repository layout and sampled entry points. Some subsystem wiring is inferred only where the README explicitly describes it; unsampled internals are not expanded.

Main entry points: [`data_processing/prepare_colmap_artifixer_inputs.py`](https://github.com/nv-tlabs/artifixer/blob/main/data_processing/prepare_colmap_artifixer_inputs.py) (COLMAP preparation), [`model_eval/run_inference.py`](https://github.com/nv-tlabs/artifixer/blob/main/model_eval/run_inference.py) (Inference runner), [`model_training/pipeline/pipeline.py`](https://github.com/nv-tlabs/artifixer/blob/main/model_training/pipeline/pipeline.py) (Diffusion pipeline), [`model_training/train.py`](https://github.com/nv-tlabs/artifixer/blob/main/model_training/train.py) (Training entry)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_prep["Scene preparation"]
  node_prepared["Prepared scene"]
  node_prepare["COLMAP preparation"]
  node_reconstruction["Sparse reconstruction<br/>[workflow.py]"]
  node_render["3DGRUT rendering"]
  node_scale["Metric alignment"]
  node_captions["Caption embeddings"]
end

subgraph group_training["Model training"]
  node_train_data["Training data<br/>[dl3dv_base.py]"]
  node_train_entry["Training entry<br/>[train.py]"]
  node_trainer["Trainer loop<br/>[trainer_base.py]"]
  node_distill["Distillation<br/>[distillation.py]"]
end

subgraph group_inference["Inference"]
  node_pipeline["Diffusion pipeline<br/>[pipeline.py]"]
  node_kv_pipeline["KV-cache pipeline"]
  node_transformer["Video transformer<br/>[transformer.py]"]
  node_vae["Video latent codec<br/>[pipeline_base.py]"]
  node_infer["Inference runner<br/>[run_inference.py]"]
  node_recon_eval["COLMAP eval data"]
  node_results["Corrected frames"]
end

subgraph group_evaluation["Evaluation"]
  node_eval_data["Evaluation datasets"]
  node_metrics["Evaluation metrics<br/>[metrics_utils.py]"]
  node_eval_dl3dv["DL3DV metrics"]
  node_eval_nerf["Nerfbusters metrics"]
end

node_user(("Engineer"))
node_colmap[("COLMAP scene")]
node_moge["MoGe model"]
node_dl3dv[("DL3DV scenes")]

node_user -->|"prepares scene"| node_prepare
node_colmap -->|"provides inputs"| node_prepare
node_prepare -->|"trains reconstruction"| node_reconstruction
node_prepare -->|"renders views"| node_render
node_prepare -->|"estimates scale"| node_scale
node_prepare -->|"generates captions"| node_captions
node_scale -.->|"estimates depth"| node_moge
node_prepare -->|"writes dataset"| node_prepared
node_dl3dv -->|"supplies scenes"| node_train_data
node_train_data -->|"loads batches"| node_train_entry
node_train_entry -->|"runs training"| node_trainer
node_trainer -->|"trains model"| node_pipeline
node_train_entry -.->|"optional distillation"| node_distill
node_user -->|"requests correction"| node_infer
node_prepared -->|"loads split"| node_recon_eval
node_infer -->|"creates dataset"| node_recon_eval
node_infer -->|"runs inference"| node_pipeline
node_pipeline -->|"denoises latents"| node_transformer
node_kv_pipeline -->|"denoises chunks"| node_transformer
node_pipeline -->|"encodes and decodes"| node_vae
node_infer -->|"writes frames"| node_results
node_eval_data -->|"provides evaluation data"| node_eval_dl3dv
node_eval_data -->|"provides evaluation data"| node_eval_nerf
node_eval_dl3dv -->|"computes scores"| node_metrics
node_eval_nerf -->|"computes scores"| node_metrics

click node_prepare "https://github.com/nv-tlabs/artifixer/blob/main/data_processing/prepare_colmap_artifixer_inputs.py"
click node_reconstruction "https://github.com/nv-tlabs/artifixer/blob/main/data_processing/sparse_recon/workflow.py"
click node_render "https://github.com/nv-tlabs/artifixer/blob/main/data_processing/render_3dgrut_colmap.py"
click node_scale "https://github.com/nv-tlabs/artifixer/blob/main/data_processing/sparse_recon/metric_alignment.py"
click node_captions "https://github.com/nv-tlabs/artifixer/blob/main/data_processing/captioning/generate_captions.py"
click node_train_data "https://github.com/nv-tlabs/artifixer/blob/main/model_training/data/dl3dv_base.py"
click node_train_entry "https://github.com/nv-tlabs/artifixer/blob/main/model_training/train.py"
click node_trainer "https://github.com/nv-tlabs/artifixer/blob/main/model_training/trainers/trainer_base.py"
click node_distill "https://github.com/nv-tlabs/artifixer/blob/main/model_training/distillation.py"
click node_pipeline "https://github.com/nv-tlabs/artifixer/blob/main/model_training/pipeline/pipeline.py"
click node_kv_pipeline "https://github.com/nv-tlabs/artifixer/blob/main/model_training/pipeline/kv_cache_pipeline.py"
click node_transformer "https://github.com/nv-tlabs/artifixer/blob/main/model_training/net/transformer.py"
click node_vae "https://github.com/nv-tlabs/artifixer/blob/main/model_training/pipeline/pipeline_base.py"
click node_infer "https://github.com/nv-tlabs/artifixer/blob/main/model_eval/run_inference.py"
click node_recon_eval "https://github.com/nv-tlabs/artifixer/blob/main/model_eval/datasets/reconstructed_colmap_eval.py"
click node_eval_data "https://github.com/nv-tlabs/artifixer/tree/main/model_eval/datasets"
click node_metrics "https://github.com/nv-tlabs/artifixer/blob/main/model_eval/metrics_utils.py"
click node_eval_dl3dv "https://github.com/nv-tlabs/artifixer/blob/main/model_eval/compute_metrics_dl3dv.py"
click node_eval_nerf "https://github.com/nv-tlabs/artifixer/blob/main/model_eval/compute_metrics_nerfbusters.py"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_prepared,node_prepare,node_reconstruction,node_render,node_scale,node_captions toneBlue
class node_train_data,node_train_entry,node_trainer,node_distill,node_colmap,node_dl3dv toneAmber
class node_pipeline,node_kv_pipeline,node_transformer,node_vae,node_infer,node_recon_eval,node_results toneMint
class node_eval_data,node_metrics,node_eval_dl3dv,node_eval_nerf toneRose
class node_user,node_moge toneIndigo
```

## Components

### Scene preparation

- **Prepared scene**
- [COLMAP preparation](https://github.com/nv-tlabs/artifixer/blob/main/data_processing/prepare_colmap_artifixer_inputs.py): `data_processing/prepare_colmap_artifixer_inputs.py`
- [Sparse reconstruction](https://github.com/nv-tlabs/artifixer/blob/main/data_processing/sparse_recon/workflow.py): `data_processing/sparse_recon/workflow.py`
- [3DGRUT rendering](https://github.com/nv-tlabs/artifixer/blob/main/data_processing/render_3dgrut_colmap.py): `data_processing/render_3dgrut_colmap.py`
- [Metric alignment](https://github.com/nv-tlabs/artifixer/blob/main/data_processing/sparse_recon/metric_alignment.py): `data_processing/sparse_recon/metric_alignment.py`
- [Caption embeddings](https://github.com/nv-tlabs/artifixer/blob/main/data_processing/captioning/generate_captions.py): `data_processing/captioning/generate_captions.py`

### Model training

- [Training data](https://github.com/nv-tlabs/artifixer/blob/main/model_training/data/dl3dv_base.py): `model_training/data/dl3dv_base.py`
- [Training entry](https://github.com/nv-tlabs/artifixer/blob/main/model_training/train.py): `model_training/train.py`
- [Trainer loop](https://github.com/nv-tlabs/artifixer/blob/main/model_training/trainers/trainer_base.py): `model_training/trainers/trainer_base.py`
- [Distillation](https://github.com/nv-tlabs/artifixer/blob/main/model_training/distillation.py): `model_training/distillation.py`

### Inference

- [Diffusion pipeline](https://github.com/nv-tlabs/artifixer/blob/main/model_training/pipeline/pipeline.py): `model_training/pipeline/pipeline.py`
- [KV-cache pipeline](https://github.com/nv-tlabs/artifixer/blob/main/model_training/pipeline/kv_cache_pipeline.py): `model_training/pipeline/kv_cache_pipeline.py`
- [Video transformer](https://github.com/nv-tlabs/artifixer/blob/main/model_training/net/transformer.py): `model_training/net/transformer.py`
- [Video latent codec](https://github.com/nv-tlabs/artifixer/blob/main/model_training/pipeline/pipeline_base.py): `model_training/pipeline/pipeline_base.py`
- [Inference runner](https://github.com/nv-tlabs/artifixer/blob/main/model_eval/run_inference.py): `model_eval/run_inference.py`
- [COLMAP eval data](https://github.com/nv-tlabs/artifixer/blob/main/model_eval/datasets/reconstructed_colmap_eval.py): `model_eval/datasets/reconstructed_colmap_eval.py`
- **Corrected frames**

### Evaluation

- [Evaluation datasets](https://github.com/nv-tlabs/artifixer/tree/main/model_eval/datasets): `model_eval/datasets`
- [Evaluation metrics](https://github.com/nv-tlabs/artifixer/blob/main/model_eval/metrics_utils.py): `model_eval/metrics_utils.py`
- [DL3DV metrics](https://github.com/nv-tlabs/artifixer/blob/main/model_eval/compute_metrics_dl3dv.py): `model_eval/compute_metrics_dl3dv.py`
- [Nerfbusters metrics](https://github.com/nv-tlabs/artifixer/blob/main/model_eval/compute_metrics_nerfbusters.py): `model_eval/compute_metrics_nerfbusters.py`

### Other components

- **Engineer**
- **COLMAP scene**
- **MoGe model**
- **DL3DV scenes**

## Connections

- Engineer → COLMAP preparation: prepares scene
- COLMAP scene → COLMAP preparation: provides inputs
- COLMAP preparation → Sparse reconstruction: trains reconstruction
- COLMAP preparation → 3DGRUT rendering: renders views
- COLMAP preparation → Metric alignment: estimates scale
- COLMAP preparation → Caption embeddings: generates captions
- Metric alignment → MoGe model: estimates depth
- COLMAP preparation → Prepared scene: writes dataset
- DL3DV scenes → Training data: supplies scenes
- Training data → Training entry: loads batches
- Training entry → Trainer loop: runs training
- Trainer loop → Diffusion pipeline: trains model
- Training entry → Distillation: optional distillation
- Engineer → Inference runner: requests correction
- Prepared scene → COLMAP eval data: loads split
- Inference runner → COLMAP eval data: creates dataset
- Inference runner → Diffusion pipeline: runs inference
- Diffusion pipeline → Video transformer: denoises latents
- KV-cache pipeline → Video transformer: denoises chunks
- Diffusion pipeline → Video latent codec: encodes and decodes
- Inference runner → Corrected frames: writes frames
- Evaluation datasets → DL3DV metrics: provides evaluation data
- Evaluation datasets → Nerfbusters metrics: provides evaluation data
- DL3DV metrics → Evaluation metrics: computes scores
- Nerfbusters metrics → Evaluation metrics: computes scores