# nianticlabs/ace architecture

> Architecture diagram of the GitHub repository nianticlabs/ace, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-10-02). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/nianticlabs/ace
- Repository: https://github.com/nianticlabs/ace

## Overview

ACE trains scene-specific coordinate-regression heads from RGB images and camera poses, then combines each head with a pretrained encoder to predict dense scene coordinates and estimate query-camera poses with DSAC* RANSAC. Dataset preparation, optional visualization, and ensemble/clustering support are additional product branches. The graph follows README workflow statements and SOURCE INDEX references; several implementations were not sampled, so uncertain internal links are omitted.

Main entry points: [`ace_trainer.py`](https://github.com/nianticlabs/ace/blob/main/ace_trainer.py) (Training orchestration), [`test_ace.py`](https://github.com/nianticlabs/ace/blob/main/test_ace.py) (Evaluation command), [`dsacstar/dsacstar.cpp`](https://github.com/nianticlabs/ace/blob/main/dsacstar/dsacstar.cpp) (RANSAC bindings), [`dataset.py`](https://github.com/nianticlabs/ace/blob/main/dataset.py) (Image and pose loader)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_data["Dataset preparation"]
  node_setup_7["7-Scenes setup<br/>[setup_7scenes.py]"]
  node_setup_12["12-Scenes setup<br/>[setup_12scenes.py]"]
  node_setup_other["Other dataset setup"]
  node_dataset_util["Dataset utilities<br/>[dataset_util.py]"]
  node_dataset["Image and pose loader<br/>[dataset.py]"]
end

subgraph group_learning["ACE training"]
  node_train_entry["Training command<br/>[train_ace.py]"]
  node_trainer["Training orchestration<br/>[ace_trainer.py]"]
  node_network["Encoder and regressor<br/>[ace_network.py]"]
  node_loss["Coordinate loss<br/>[ace_loss.py]"]
  node_encoder_weights["Pretrained encoder"]
  node_head_weights["Scene head weights"]
  node_util["ACE utilities<br/>[ace_util.py]"]
end

subgraph group_localization["Pose estimation"]
  node_test_entry["Evaluation command<br/>[test_ace.py]"]
  node_eval_poses["Pose evaluation metrics<br/>[eval_poses.py]"]
  node_coordinates["Scene coordinates"]
end

subgraph group_native["DSAC* engine"]
  node_dsac_cpp["RANSAC bindings<br/>[dsacstar.cpp]"]
  node_dsac_util["Hypotheses and scoring<br/>[dsacstar_util.h]"]
  node_dsac_derivative["Pose derivatives"]
  node_dsac_loss["RANSAC loss<br/>[dsacstar_loss.h]"]
  node_dsac_types["Pose and coordinate types<br/>[dsacstar_types.h]"]
  node_thread_rand["Thread random numbers<br/>[thread_rand.h]"]
  node_stop_watch["Timing utility<br/>[stop_watch.h]"]
end

subgraph group_visual["Visualization"]
  node_visualizer["Training and test rendering<br/>[ace_visualizer.py]"]
  node_vis_util["Rendering geometry<br/>[ace_vis_util.py]"]
end

node_user(("Researcher"))
node_scene_data["Scene data"]

node_user -->|"prepares data"| node_setup_7
node_user -->|"prepares data"| node_setup_12
node_user -->|"prepares data"| node_setup_other
node_setup_7 -->|"uses helpers"| node_dataset_util
node_setup_12 -->|"uses helpers"| node_dataset_util
node_setup_other -.->|"uses helpers"| node_dataset_util
node_scene_data -->|"loads images and poses"| node_dataset
node_user -->|"starts training"| node_train_entry
node_train_entry -->|"invokes"| node_trainer
node_trainer -->|"loads samples"| node_dataset
node_trainer -->|"trains regressor"| node_network
node_trainer -->|"computes loss"| node_loss
node_trainer -->|"loads weights"| node_encoder_weights
node_trainer -->|"saves head"| node_head_weights
node_trainer -->|"renders progress"| node_visualizer
node_visualizer -->|"uses geometry"| node_vis_util
node_user -->|"starts evaluation"| node_test_entry
node_test_entry -->|"loads query frames"| node_dataset
node_test_entry -->|"predicts coordinates"| node_network
node_test_entry -->|"loads head"| node_head_weights
node_test_entry -->|"loads encoder"| node_encoder_weights
node_network -->|"predicts"| node_coordinates
node_test_entry -->|"estimates poses"| node_dsac_cpp
node_test_entry -->|"evaluates results"| node_eval_poses
node_test_entry -.->|"renders results"| node_visualizer
node_dsac_cpp -->|"calls routines"| node_dsac_util
node_dsac_cpp -->|"uses derivatives"| node_dsac_derivative
node_dsac_cpp -->|"uses loss"| node_dsac_loss
node_dsac_cpp -->|"uses types"| node_dsac_types
node_dsac_cpp -->|"uses random source"| node_thread_rand
node_dsac_cpp -->|"times operations"| node_stop_watch
node_dsac_util -->|"samples randomly"| node_thread_rand
node_eval_poses -->|"reads dataset poses"| node_dataset
node_dataset -->|"provides model inputs"| node_network
node_trainer -->|"uses helpers"| node_util

click node_setup_7 "https://github.com/nianticlabs/ace/blob/main/datasets/setup_7scenes.py"
click node_setup_12 "https://github.com/nianticlabs/ace/blob/main/datasets/setup_12scenes.py"
click node_setup_other "https://github.com/nianticlabs/ace/tree/main/datasets"
click node_dataset_util "https://github.com/nianticlabs/ace/blob/main/datasets/dataset_util.py"
click node_dataset "https://github.com/nianticlabs/ace/blob/main/dataset.py"
click node_train_entry "https://github.com/nianticlabs/ace/blob/main/train_ace.py"
click node_trainer "https://github.com/nianticlabs/ace/blob/main/ace_trainer.py"
click node_network "https://github.com/nianticlabs/ace/blob/main/ace_network.py"
click node_loss "https://github.com/nianticlabs/ace/blob/main/ace_loss.py"
click node_encoder_weights "https://github.com/nianticlabs/ace/blob/main/ace_encoder_pretrained.pt"
click node_test_entry "https://github.com/nianticlabs/ace/blob/main/test_ace.py"
click node_eval_poses "https://github.com/nianticlabs/ace/blob/main/eval_poses.py"
click node_dsac_cpp "https://github.com/nianticlabs/ace/blob/main/dsacstar/dsacstar.cpp"
click node_dsac_util "https://github.com/nianticlabs/ace/blob/main/dsacstar/dsacstar_util.h"
click node_dsac_derivative "https://github.com/nianticlabs/ace/blob/main/dsacstar/dsacstar_derivative.h"
click node_dsac_loss "https://github.com/nianticlabs/ace/blob/main/dsacstar/dsacstar_loss.h"
click node_dsac_types "https://github.com/nianticlabs/ace/blob/main/dsacstar/dsacstar_types.h"
click node_thread_rand "https://github.com/nianticlabs/ace/blob/main/dsacstar/thread_rand.h"
click node_stop_watch "https://github.com/nianticlabs/ace/blob/main/dsacstar/stop_watch.h"
click node_visualizer "https://github.com/nianticlabs/ace/blob/main/ace_visualizer.py"
click node_vis_util "https://github.com/nianticlabs/ace/blob/main/ace_vis_util.py"
click node_util "https://github.com/nianticlabs/ace/blob/main/ace_util.py"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_setup_7,node_setup_12,node_setup_other,node_dataset_util,node_dataset toneBlue
class node_train_entry,node_trainer,node_network,node_loss,node_encoder_weights,node_head_weights,node_util toneAmber
class node_test_entry,node_eval_poses,node_coordinates toneMint
class node_dsac_cpp,node_dsac_util,node_dsac_derivative,node_dsac_loss,node_dsac_types,node_thread_rand,node_stop_watch toneRose
class node_visualizer,node_vis_util,node_user,node_scene_data toneIndigo
```

## Components

### Dataset preparation

- [7-Scenes setup](https://github.com/nianticlabs/ace/blob/main/datasets/setup_7scenes.py): `datasets/setup_7scenes.py`
- [12-Scenes setup](https://github.com/nianticlabs/ace/blob/main/datasets/setup_12scenes.py): `datasets/setup_12scenes.py`
- [Other dataset setup](https://github.com/nianticlabs/ace/tree/main/datasets): `datasets`
- [Dataset utilities](https://github.com/nianticlabs/ace/blob/main/datasets/dataset_util.py): `datasets/dataset_util.py`
- [Image and pose loader](https://github.com/nianticlabs/ace/blob/main/dataset.py): `dataset.py`

### ACE training

- [Training command](https://github.com/nianticlabs/ace/blob/main/train_ace.py): `train_ace.py`
- [Training orchestration](https://github.com/nianticlabs/ace/blob/main/ace_trainer.py): `ace_trainer.py`
- [Encoder and regressor](https://github.com/nianticlabs/ace/blob/main/ace_network.py): `ace_network.py`
- [Coordinate loss](https://github.com/nianticlabs/ace/blob/main/ace_loss.py): `ace_loss.py`
- [Pretrained encoder](https://github.com/nianticlabs/ace/blob/main/ace_encoder_pretrained.pt): `ace_encoder_pretrained.pt`
- **Scene head weights**
- [ACE utilities](https://github.com/nianticlabs/ace/blob/main/ace_util.py): `ace_util.py`

### Pose estimation

- [Evaluation command](https://github.com/nianticlabs/ace/blob/main/test_ace.py): `test_ace.py`
- [Pose evaluation metrics](https://github.com/nianticlabs/ace/blob/main/eval_poses.py): `eval_poses.py`
- **Scene coordinates**

### DSAC* engine

- [RANSAC bindings](https://github.com/nianticlabs/ace/blob/main/dsacstar/dsacstar.cpp): `dsacstar/dsacstar.cpp`
- [Hypotheses and scoring](https://github.com/nianticlabs/ace/blob/main/dsacstar/dsacstar_util.h): `dsacstar/dsacstar_util.h`
- [Pose derivatives](https://github.com/nianticlabs/ace/blob/main/dsacstar/dsacstar_derivative.h): `dsacstar/dsacstar_derivative.h`
- [RANSAC loss](https://github.com/nianticlabs/ace/blob/main/dsacstar/dsacstar_loss.h): `dsacstar/dsacstar_loss.h`
- [Pose and coordinate types](https://github.com/nianticlabs/ace/blob/main/dsacstar/dsacstar_types.h): `dsacstar/dsacstar_types.h`
- [Thread random numbers](https://github.com/nianticlabs/ace/blob/main/dsacstar/thread_rand.h): `dsacstar/thread_rand.h`
- [Timing utility](https://github.com/nianticlabs/ace/blob/main/dsacstar/stop_watch.h): `dsacstar/stop_watch.h`

### Visualization

- [Training and test rendering](https://github.com/nianticlabs/ace/blob/main/ace_visualizer.py): `ace_visualizer.py`
- [Rendering geometry](https://github.com/nianticlabs/ace/blob/main/ace_vis_util.py): `ace_vis_util.py`

### Other components

- **Researcher**
- **Scene data**

## Connections

- Researcher → 7-Scenes setup: prepares data (evidence: `README.md`)
- Researcher → 12-Scenes setup: prepares data (evidence: `README.md`)
- Researcher → Other dataset setup: prepares data (evidence: `README.md`)
- 7-Scenes setup → Dataset utilities: uses helpers (evidence: `datasets/setup_7scenes.py`)
- 12-Scenes setup → Dataset utilities: uses helpers (evidence: `datasets/setup_12scenes.py`)
- Other dataset setup → Dataset utilities: uses helpers (evidence: `datasets/setup_7scenes.py`)
- Scene data → Image and pose loader: loads images and poses (evidence: `README.md`)
- Researcher → Training command: starts training (evidence: `README.md`)
- Training command → Training orchestration: invokes (evidence: `train_ace.py`)
- Training orchestration → Image and pose loader: loads samples (evidence: `ace_trainer.py`)
- Training orchestration → Encoder and regressor: trains regressor (evidence: `ace_trainer.py`)
- Training orchestration → Coordinate loss: computes loss (evidence: `ace_trainer.py`)
- Training orchestration → Pretrained encoder: loads weights (evidence: `README.md`)
- Training orchestration → Scene head weights: saves head (evidence: `README.md`)
- Training orchestration → Training and test rendering: renders progress (evidence: `ace_trainer.py`)
- Training and test rendering → Rendering geometry: uses geometry (evidence: `ace_visualizer.py`)
- Researcher → Evaluation command: starts evaluation (evidence: `README.md`)
- Evaluation command → Image and pose loader: loads query frames (evidence: `README.md`)
- Evaluation command → Encoder and regressor: predicts coordinates (evidence: `README.md`)
- Evaluation command → Scene head weights: loads head (evidence: `README.md`)
- Evaluation command → Pretrained encoder: loads encoder (evidence: `README.md`)
- Encoder and regressor → Scene coordinates: predicts (evidence: `README.md`)
- Evaluation command → RANSAC bindings: estimates poses (evidence: `README.md`)
- Evaluation command → Pose evaluation metrics: evaluates results (evidence: `README.md`)
- Evaluation command → Training and test rendering: renders results
- RANSAC bindings → Hypotheses and scoring: calls routines (evidence: `dsacstar/dsacstar.cpp`)
- RANSAC bindings → Pose derivatives: uses derivatives (evidence: `dsacstar/dsacstar.cpp`)
- RANSAC bindings → RANSAC loss: uses loss (evidence: `dsacstar/dsacstar.cpp`)
- RANSAC bindings → Pose and coordinate types: uses types (evidence: `dsacstar/dsacstar.cpp`)
- RANSAC bindings → Thread random numbers: uses random source (evidence: `dsacstar/dsacstar.cpp`)
- RANSAC bindings → Timing utility: times operations (evidence: `dsacstar/dsacstar.cpp`)
- Hypotheses and scoring → Thread random numbers: samples randomly (evidence: `dsacstar/dsacstar_util.h`)
- Pose evaluation metrics → Image and pose loader: reads dataset poses (evidence: `eval_poses.py`)
- Image and pose loader → Encoder and regressor: provides model inputs (evidence: `dataset.py`)
- Training orchestration → ACE utilities: uses helpers (evidence: `ace_trainer.py`)
