# junyi42/loger architecture

> Architecture diagram of the GitHub repository junyi42/loger, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-04-01). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/junyi42/loger
- Repository: https://github.com/junyi42/loger

## Overview

1. **What this repo is**
   - LoGeR is a PyTorch-based reimplementation of a long-video geometric reconstruction system.
   - The central architectural idea is **chunked video processing with hybrid memory**: frames are handled in segments, and state is carried across chunks to preserve long-range geometric consistency.

2. **Core model boundary**
   - The main model code lives in `loger/models/`.
   - `pi3.py`, `pi3x.py`, and `ttt.py` are the top-level model variants/adaptations.
   - This is the boundary where raw image/video inputs become geometry outputs such as depth, pose, or reconstruction features.

3. **Backbone and feature extraction**
   - `loger/models/dinov2/` vendors a DINOv2-style vision transformer backbone.
   - It includes the transformer blocks, patch embedding, attention, MLP, heads, and config/utility code needed to run the backbone inside this repo.
   - This backbone is the feature encoder feeding the downstream geometric heads.

4. **Task-specific heads and geometric prediction**
   - `loger/models/layers/` contains specialized output modules:
     - `camera_head.py` for camera/pose-related prediction,
     - `conv_head.py` and `transformer_head.py` for alternative decoding paths,
     - `pos_embed.py`, `attention.py`, and `block.py` for transformer internals,
     - `prope.py` for additional positional/propagation logic.
   - These modules form the bridge from visual features to geometry-aware outputs.

5. **Geometry and rotation utilities**
   - `loger/utils/geometry.py`, `rotation.py`, and `basic.py` provide the math used across model and evaluation code.
   - These utilities are the shared layer for coordinate transforms, pose handling, and geometric conversions.

6. **Checkpoint and configuration boundary**
   - Pretrained weights are stored outside the model code under `ckpts/LoGeR/` and `ckpts/LoGeR_star/`.
   - Each checkpoint folder has an `original_config.yaml`, indicating that runtime behavior depends on paired model weights plus configuration.
   - The repo expects `latest.pt` in these directories, so loading is a distinct runtime boundary from code.

7. **Demo/visualization path**
   - `demo_run.sh` and `demo_viser.py` are the interactive demo entry points.
   - `loger/utils/viser_utils.py` and `visual_util.py` suggest a visualization pipeline built around **Viser** for inspecting outputs.
   - The demo likely connects sample data from `data/examples/office` to the model and renders reconstruction results interactively.

8. **Evaluation as a separate subsystem**
   - The `eval/` tree is not just test code; it is a full benchmark harness with multiple task-specific pipelines.
   - It contains its own launch scripts, metadata, dataset preprocessing, metrics, and adapters.
   - This means the repo has a clear split between **core inference code** and **evaluation orchestration**.

9. **Video depth and relative pose evaluation**
   - `eval/video_depth/` handles depth estimation benchmarks, with dataset-specific run scripts for Bonn, KITTI, and Sintel.
   - `eval/relpose/` handles relative pose evaluation, including sequence reuse logic and EVO-based scoring utilities.
   - These subsystems consume model predictions and compare them against benchmark ground truth with task-specific metrics.

10. **Multiview reconstruction and long-benchmark tooling**
   - `eval/mv_recon/` is a self-contained multiview reconstruction evaluation stack with its own data pipeline, criterion, metric utilities, and device/parallel helpers.
   - `eval/long_eval_script/` contains C++ benchmark code for KITTI and VBR, indicating some evaluations rely on compiled external benchmark executables.
   - `eval/datasets_preprocess/` standardizes datasets such as TUM, ScanNet, and Bonn before evaluation.

11. **External model and benchmark integration**
   - The README notes the code is based on Pi3 and LaCT, and evaluation borrows from TTT3R, VBR, and Viser.
   - `eval/pi3_adapter.py` is a compatibility layer that likely bridges LoGeR outputs to legacy Pi3-style evaluation or interfaces.
   - Practically, this repo sits at the intersection of a custom long-context model and several established geometry benchmarks.

12. **Overall data flow**
   - Input videos/images enter through demo or eval entry points.
   - Data is chunked, encoded by the DINOv2 backbone, processed by LoGeR’s memory-aware model, and decoded by geometry heads.
   - Outputs then branch into either visualization/demo rendering or one of the evaluation pipelines, where dataset-specific metrics and benchmark scripts consume them.

Main entry points: [`loger/models/pi3.py`](https://github.com/junyi42/loger/blob/main/loger/models/pi3.py) (PI3), [`demo_viser.py`](https://github.com/junyi42/loger/blob/main/demo_viser.py) (Demo run), [`loger/models/layers`](https://github.com/junyi42/loger/tree/main/loger/models/layers) (Geom heads), [`loger/models/dinov2/models/vision_transformer.py`](https://github.com/junyi42/loger/blob/main/loger/models/dinov2/models/vision_transformer.py) (DINOv2)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_core["Core model"]
  node_pi3["PI3<br/>model variant<br/>[pi3.py]"]
  node_pi3x["PI3X<br/>model variant<br/>[pi3x.py]"]
  node_ttt["TTT<br/>model variant<br/>[ttt.py]"]
  node_dinov2["DINOv2<br/>vision backbone"]
  node_dinolayers["ViT layers<br/>backbone internals"]
  node_model_layers["Geom heads<br/>decoding heads"]
  node_model_attn["Model attention<br/>model internals<br/>[attention.py]"]
end

subgraph group_eval["Evaluation"]
  node_mv_recon["MV recon<br/>evaluation stack"]
  node_video_depth["Depth eval<br/>benchmark suite"]
  node_relpose["Rel pose<br/>benchmark suite"]
  node_preprocess["Preprocess<br/>dataset prep"]
  node_long_bench["Long bench<br/>native benchmark"]
  node_pi3_adapter["Pi3 adapter<br/>compat layer<br/>[pi3_adapter.py]"]
end

subgraph group_demo["Demo & viz"]
  node_vis_utils["Viz utils<br/>visualization support<br/>[viser_utils.py]"]
  node_demo_entry["Demo run<br/>entry point<br/>[demo_viser.py]"]
  node_demo_shell["Demo script<br/>launcher<br/>[demo_run.sh]"]
end

subgraph group_assets["Assets"]
  node_checkpoint_bundle[("Checkpoints<br/>weights+config")]
  node_example_data["Example data<br/>sample inputs"]
end

node_geometry_utils["Geometry utils<br/>math utilities<br/>[geometry.py]"]
node_basic_utils["Basic utils<br/>shared helpers<br/>[basic.py]"]

node_demo_shell -->|"launches"| node_demo_entry
node_demo_entry -->|"loads"| node_example_data
node_demo_entry -->|"loads"| node_checkpoint_bundle
node_demo_entry -->|"runs"| node_pi3
node_pi3 -->|"encodes"| node_dinov2
node_pi3 -->|"decodes"| node_model_layers
node_pi3 -->|"uses"| node_geometry_utils
node_pi3x -->|"encodes"| node_dinov2
node_pi3x -->|"decodes"| node_model_layers
node_ttt -->|"encodes"| node_dinov2
node_ttt -->|"decodes"| node_model_layers
node_dinov2 -->|"built from"| node_dinolayers
node_model_layers -->|"includes"| node_model_attn
node_model_layers -->|"uses"| node_geometry_utils
node_mv_recon -->|"consumes"| node_pi3
node_video_depth -->|"consumes"| node_pi3
node_relpose -->|"consumes"| node_pi3
node_preprocess -->|"preps for"| node_video_depth
node_preprocess -->|"preps for"| node_relpose
node_long_bench -->|"supports"| node_mv_recon
node_pi3_adapter -->|"adapts"| node_relpose
node_vis_utils -->|"renders"| node_demo_entry
node_checkpoint_bundle -.->|"configures"| node_pi3

click node_pi3 "https://github.com/junyi42/loger/blob/main/loger/models/pi3.py"
click node_pi3x "https://github.com/junyi42/loger/blob/main/loger/models/pi3x.py"
click node_ttt "https://github.com/junyi42/loger/blob/main/loger/models/ttt.py"
click node_dinov2 "https://github.com/junyi42/loger/blob/main/loger/models/dinov2/models/vision_transformer.py"
click node_dinolayers "https://github.com/junyi42/loger/tree/main/loger/models/dinov2/layers"
click node_model_layers "https://github.com/junyi42/loger/tree/main/loger/models/layers"
click node_model_attn "https://github.com/junyi42/loger/blob/main/loger/models/layers/attention.py"
click node_geometry_utils "https://github.com/junyi42/loger/blob/main/loger/utils/geometry.py"
click node_basic_utils "https://github.com/junyi42/loger/blob/main/loger/utils/basic.py"
click node_vis_utils "https://github.com/junyi42/loger/blob/main/loger/utils/viser_utils.py"
click node_demo_entry "https://github.com/junyi42/loger/blob/main/demo_viser.py"
click node_demo_shell "https://github.com/junyi42/loger/blob/main/demo_run.sh"
click node_checkpoint_bundle "https://github.com/junyi42/loger/tree/main/ckpts/LoGeR"
click node_example_data "https://github.com/junyi42/loger/tree/main/data/examples/office"
click node_mv_recon "https://github.com/junyi42/loger/tree/main/eval/mv_recon"
click node_video_depth "https://github.com/junyi42/loger/tree/main/eval/video_depth"
click node_relpose "https://github.com/junyi42/loger/tree/main/eval/relpose"
click node_preprocess "https://github.com/junyi42/loger/tree/main/eval/datasets_preprocess"
click node_long_bench "https://github.com/junyi42/loger/tree/main/eval/long_eval_script"
click node_pi3_adapter "https://github.com/junyi42/loger/blob/main/eval/pi3_adapter.py"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_pi3,node_pi3x,node_ttt,node_dinov2,node_dinolayers,node_model_layers,node_model_attn toneBlue
class node_mv_recon,node_video_depth,node_relpose,node_preprocess,node_long_bench,node_pi3_adapter toneAmber
class node_vis_utils,node_demo_entry,node_demo_shell toneMint
class node_checkpoint_bundle,node_example_data toneRose
class node_geometry_utils,node_basic_utils toneNeutral
```

## Components

### Core model

Long-video reconstruction model, backbone, and geometry heads.

- [PI3](https://github.com/junyi42/loger/blob/main/loger/models/pi3.py): model variant · Main long-video reconstruction model variant. · `loger/models/pi3.py`
- [PI3X](https://github.com/junyi42/loger/blob/main/loger/models/pi3x.py): model variant · Alternative model adaptation for long-context processing. · `loger/models/pi3x.py`
- [TTT](https://github.com/junyi42/loger/blob/main/loger/models/ttt.py): model variant · Third top-level model variant/adaptation. · `loger/models/ttt.py`
- [DINOv2](https://github.com/junyi42/loger/blob/main/loger/models/dinov2/models/vision_transformer.py): vision backbone · Vendored transformer backbone for visual feature extraction. · `loger/models/dinov2/models/vision_transformer.py`
- [ViT layers](https://github.com/junyi42/loger/tree/main/loger/models/dinov2/layers): backbone internals · Attention, MLP, patch embedding, and transformer blocks for the backbone. · `loger/models/dinov2/layers`
- [Geom heads](https://github.com/junyi42/loger/tree/main/loger/models/layers): decoding heads · Camera, convolutional, and transformer heads that map features to geometry outputs. · `loger/models/layers`
- [Model attention](https://github.com/junyi42/loger/blob/main/loger/models/layers/attention.py): model internals · Shared transformer attention, blocks, and positional modules used by the decoder side. · `loger/models/layers/attention.py`

### Evaluation

Benchmark harnesses and task-specific evaluation pipelines.

- [MV recon](https://github.com/junyi42/loger/tree/main/eval/mv_recon): evaluation stack · Self-contained multiview reconstruction benchmark pipeline. · `eval/mv_recon`
- [Depth eval](https://github.com/junyi42/loger/tree/main/eval/video_depth): benchmark suite · Video depth evaluation across KITTI, Bonn, and Sintel. · `eval/video_depth`
- [Rel pose](https://github.com/junyi42/loger/tree/main/eval/relpose): benchmark suite · Relative pose evaluation with sequence reuse and EVO scoring. · `eval/relpose`
- [Preprocess](https://github.com/junyi42/loger/tree/main/eval/datasets_preprocess): dataset prep · Dataset standardization scripts for long-form evaluation inputs. · `eval/datasets_preprocess`
- [Long bench](https://github.com/junyi42/loger/tree/main/eval/long_eval_script): native benchmark · Compiled benchmark executables for KITTI and VBR evaluation. · `eval/long_eval_script`
- [Pi3 adapter](https://github.com/junyi42/loger/blob/main/eval/pi3_adapter.py): compat layer · Bridges LoGeR outputs to Pi3-style evaluation or interfaces. · `eval/pi3_adapter.py`

### Demo & viz

Interactive demo entry points and visualization support.

- [Viz utils](https://github.com/junyi42/loger/blob/main/loger/utils/viser_utils.py): visualization support · Rendering and Viser integration utilities for interactive inspection. · `loger/utils/viser_utils.py`
- [Demo run](https://github.com/junyi42/loger/blob/main/demo_viser.py): entry point · Interactive demo script that drives inference and visualization. · `demo_viser.py`
- [Demo script](https://github.com/junyi42/loger/blob/main/demo_run.sh): launcher · Shell entry point for running the demo workflow. · `demo_run.sh`

### Assets

Pretrained checkpoints and example inputs.

- [Checkpoints](https://github.com/junyi42/loger/tree/main/ckpts/LoGeR): weights+config · Pretrained weights paired with original configuration files. · `ckpts/LoGeR`
- [Example data](https://github.com/junyi42/loger/tree/main/data/examples/office): sample inputs · Sample office scene used by the demo pipeline. · `data/examples/office`

### Other components

- [Geometry utils](https://github.com/junyi42/loger/blob/main/loger/utils/geometry.py): math utilities · Coordinate transforms, rotation handling, and shared geometric math. · `loger/utils/geometry.py`
- [Basic utils](https://github.com/junyi42/loger/blob/main/loger/utils/basic.py): shared helpers · General-purpose helper routines used across model and evaluation code. · `loger/utils/basic.py`

## Connections

- Demo script → Demo run: launches · Shell script starts the interactive demo.
- Demo run → Example data: loads · Demo consumes sample inputs from the example scene.
- Demo run → Checkpoints: loads · Demo depends on pretrained weights and config.
- Demo run → PI3: runs · Interactive demo invokes the core model variant.
- PI3 → DINOv2: encodes · Model uses the backbone for visual feature extraction.
- PI3 → Geom heads: decodes · Model uses geometry heads to produce outputs.
- PI3 → Geometry utils: uses · Model relies on shared geometry math.
- PI3X → DINOv2: encodes · Variant reuses the same feature backbone.
- PI3X → Geom heads: decodes · Variant reuses geometry-aware heads.
- TTT → DINOv2: encodes · Variant reuses the same feature backbone.
- TTT → Geom heads: decodes · Variant reuses geometry-aware heads.
- DINOv2 → ViT layers: built from · Vision transformer is assembled from internal backbone layers.
- Geom heads → Model attention: includes · Decoder-side modules include attention and transformer internals.
- Geom heads → Geometry utils: uses · Heads use geometric conversions and pose math.
- MV recon → PI3: consumes · Evaluation stack runs model predictions through multiview metrics.
- Depth eval → PI3: consumes · Depth benchmarks evaluate model outputs on video datasets.
- Rel pose → PI3: consumes · Relative pose benchmarks evaluate model predictions.
- Preprocess → Depth eval: preps for · Dataset preprocessing feeds depth benchmark pipelines.
- Preprocess → Rel pose: preps for · Dataset preprocessing feeds pose benchmark pipelines.
- Long bench → MV recon: supports · Native benchmark executables support long-form evaluation.
- Pi3 adapter → Rel pose: adapts · Compatibility layer helps bridge to legacy evaluation interfaces.
- Viz utils → Demo run: renders · Visualization utilities support the interactive demo.
- Checkpoints → PI3: configures · Checkpoint folders pair weights with the expected runtime config.
