# tomtomtommi/liteanystereo architecture

> Architecture diagram of the GitHub repository tomtomtommi/liteanystereo, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-06-28). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/tomtomtommi/liteanystereo
- Repository: https://github.com/tomtomtommi/liteanystereo

## Overview

1. **Repository purpose**
   This is an inference/evaluation codebase for the **Lite Any Stereo** family, not a training-first project. The main architectural concern is selecting a LAS variant, loading a checkpoint, running stereo matching, and reporting either visual output, benchmark metrics, MACs, or runtime.

2. **Top-level execution entry points**
   The repo is organized around a small set of scripts:
   - `demo.py` for single stereo-pair inference and visualization
   - `evaluate_stereo.py` for benchmark evaluation on standard stereo datasets
   - `evaluate.sh` as a convenience wrapper for the paper’s benchmark runs
   - `flops_count.py` for model complexity reporting
   - `profile_speed.py` for runtime measurement  
   These scripts are the main “application layer” and all delegate into the `core/` package.

3. **Model family boundary**
   The `core/` package contains the actual stereo networks. There are two model generations:
   - **LAS1** in `core/liteanystereo.py`
   - **LAS2** in `core/liteanystereov2.py`, with size variants **S/M/L/H**
   - A separate `core/liteanystereov2_H.py` suggests a specialized high-capacity variant implementation
   The README indicates LAS2 defaults to the **M** model unless `--model_size` is specified.

4. **Shared model building blocks**
   Several files indicate reusable neural-network subsystems:
   - `core/fnet.py` likely implements the feature extraction network
   - `core/aggregation.py` and `core/aggregation_fasternet.py` likely implement disparity cost aggregation / refinement
   - `core/submodule.py` likely holds shared layers, blocks, and helper modules used across LAS1/LAS2
   - `core/models.py` likely acts as the model factory or top-level wrapper that chooses the right architecture from CLI arguments  
   Architecturally, these form the internal decomposition of “feature extraction → matching/aggregation → disparity regression/output”.

5. **Inference data flow**
   The primary runtime path is stereo pair input → preprocessing → model forward pass → disparity output.  
   In the demo path, the outputs are written as:
   - a disparity visualization
   - a raw disparity array
   - optional point-cloud artifacts  
   This means the repo’s output boundary is not only numeric prediction but also visualization/export for downstream use.

6. **Dataset and evaluation subsystem**
   `core/stereo_datasets.py` is the dataset boundary for benchmark evaluation. The README shows support for:
   - `middlebury_F`
   - `middlebury_H`
   - `middlebury_Q`
   - `eth3d`
   - `kitti`
   - `drivingstereo`  
   `evaluate_stereo.py` likely builds the chosen dataset loader, runs inference, and computes benchmark-specific metrics. This is the main path for reproducing paper results.

7. **Utility layer**
   `core/utils/` and the root-level `Utils.py` provide support code around image/frame loading and general helpers:
   - `core/utils/frame_utils.py` likely handles reading stereo images, disparity maps, and related file formats
   - `core/utils/utils.py` likely contains common math, padding, normalization, logging, or evaluation helpers  
   These utilities sit below the model and dataset layers and keep the entry-point scripts small.

8. **Assets and external artifacts**
   `assets/` contains example stereo image pairs and a performance figure used by the demo and README.  
   Checkpoints are explicitly external, stored under `./checkpoints/` and fetched from Hugging Face or Google Drive. This means model weights are not part of the code path itself but are a required runtime dependency.

9. **Measurement and profiling tools**
   Two scripts support non-functional analysis:
   - `flops_count.py` measures MACs / model complexity
   - `profile_speed.py` measures inference runtime, including CUDA synchronization on GPU  
   These scripts likely instantiate the same model factory as demo/evaluation, but stop at measurement rather than producing disparities.

10. **Technology/runtime boundaries**
    The repo is centered on **PyTorch-style model inference** for stereo vision, with command-line Python entry points and shell-based evaluation orchestration. The external boundaries that matter architecturally are:
    - pretrained checkpoints
    - benchmark datasets
    - GPU execution for runtime profiling and practical deployment  
    There is no sign of a server, web UI, or training pipeline in this repository; its architecture is focused on **offline inference, benchmarking, and profiling**.

11. **How to draw it**
    A good architecture diagram would show:
    - CLI scripts at the top
    - a shared `core/` model factory beneath them
    - model variants branching into LAS1 and LAS2
    - shared feature/aggregation modules underneath the variants
    - dataset loaders and utility I/O on the side feeding inputs into the model
    - outputs flowing to visualization files, benchmark metrics, or profiling reports
    - checkpoints and datasets as external dependencies outside the repo boundary

Main entry points: [`core/models.py`](https://github.com/tomtomtommi/liteanystereo/blob/main/core/models.py) (models.py), [`demo.py`](https://github.com/tomtomtommi/liteanystereo/blob/main/demo.py) (demo.py), [`evaluate_stereo.py`](https://github.com/tomtomtommi/liteanystereo/blob/main/evaluate_stereo.py) (evaluate_stereo.py), [`core/liteanystereo.py`](https://github.com/tomtomtommi/liteanystereo/blob/main/core/liteanystereo.py) (LAS1)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_app["Entry points"]
  node_demo["demo.py<br/>CLI demo<br/>[demo.py]"]
  node_eval_stereo["evaluate_stereo.py<br/>CLI eval<br/>[evaluate_stereo.py]"]
  node_eval_sh["evaluate.sh<br/>shell wrapper<br/>[evaluate.sh]"]
  node_flops["flops_count.py<br/>MACs tool<br/>[flops_count.py]"]
  node_speed["profile_speed.py<br/>runtime profiler<br/>[profile_speed.py]"]
end

subgraph group_core["Core models"]
  node_models["models.py<br/>model factory<br/>[models.py]"]
  node_las1["LAS1<br/>stereo model<br/>[liteanystereo.py]"]
  node_las2["LAS2<br/>stereo model<br/>[liteanystereov2.py]"]
  node_las2h["LAS2-H<br/>stereo model"]
  node_fnet["fnet.py<br/>feature net<br/>[fnet.py]"]
  node_agg["aggregation.py<br/>[aggregation.py]"]
  node_agg_fast["aggregation_fasternet.py<br/>fast aggregation"]
  node_submodule["submodule.py<br/>shared blocks<br/>[submodule.py]"]
end

subgraph group_support["Support code"]
  node_datasets["stereo_datasets.py<br/>dataset loaders<br/>[stereo_datasets.py]"]
  node_frame_utils["frame_utils.py<br/>image I/O<br/>[frame_utils.py]"]
  node_utils["utils.py<br/>helpers<br/>[utils.py]"]
  node_root_utils["Utils.py<br/>top-level helpers<br/>[Utils.py]"]
end

subgraph group_external["External inputs"]
  node_checkpoints[("checkpoints<br/>weights")]
  node_datasets_ext[("benchmarks<br/>datasets")]
  node_assets["assets<br/>examples"]
end

node_demo -->|"build model"| node_models
node_demo -->|"load input"| node_frame_utils
node_demo -->|"pre/postprocess"| node_utils
node_demo -->|"sample pairs"| node_assets
node_demo -->|"load weights"| node_checkpoints
node_eval_stereo -->|"build model"| node_models
node_eval_stereo -->|"load benchmark"| node_datasets
node_eval_stereo -->|"evaluate on"| node_datasets_ext
node_eval_stereo -->|"load weights"| node_checkpoints
node_eval_sh -->|"wraps"| node_eval_stereo
node_flops -->|"instantiate"| node_models
node_speed -->|"instantiate"| node_models
node_models -->|"selects"| node_las1
node_models -->|"selects"| node_las2
node_models -->|"selects"| node_las2h
node_las1 -->|"uses"| node_fnet
node_las1 -->|"uses"| node_agg
node_las1 -->|"shares blocks"| node_submodule
node_las2 -->|"uses"| node_fnet
node_las2 -->|"uses"| node_agg_fast
node_las2 -->|"shares blocks"| node_submodule
node_las2h -->|"uses"| node_fnet
node_las2h -->|"uses"| node_agg
node_las2h -->|"shares blocks"| node_submodule
node_datasets -->|"reads data"| node_frame_utils
node_datasets -->|"helpers"| node_utils
node_frame_utils -->|"supports"| node_root_utils

click node_demo "https://github.com/tomtomtommi/liteanystereo/blob/main/demo.py"
click node_eval_stereo "https://github.com/tomtomtommi/liteanystereo/blob/main/evaluate_stereo.py"
click node_eval_sh "https://github.com/tomtomtommi/liteanystereo/blob/main/evaluate.sh"
click node_flops "https://github.com/tomtomtommi/liteanystereo/blob/main/flops_count.py"
click node_speed "https://github.com/tomtomtommi/liteanystereo/blob/main/profile_speed.py"
click node_models "https://github.com/tomtomtommi/liteanystereo/blob/main/core/models.py"
click node_las1 "https://github.com/tomtomtommi/liteanystereo/blob/main/core/liteanystereo.py"
click node_las2 "https://github.com/tomtomtommi/liteanystereo/blob/main/core/liteanystereov2.py"
click node_las2h "https://github.com/tomtomtommi/liteanystereo/blob/main/core/liteanystereov2_H.py"
click node_fnet "https://github.com/tomtomtommi/liteanystereo/blob/main/core/fnet.py"
click node_agg "https://github.com/tomtomtommi/liteanystereo/blob/main/core/aggregation.py"
click node_agg_fast "https://github.com/tomtomtommi/liteanystereo/blob/main/core/aggregation_fasternet.py"
click node_submodule "https://github.com/tomtomtommi/liteanystereo/blob/main/core/submodule.py"
click node_datasets "https://github.com/tomtomtommi/liteanystereo/blob/main/core/stereo_datasets.py"
click node_frame_utils "https://github.com/tomtomtommi/liteanystereo/blob/main/core/utils/frame_utils.py"
click node_utils "https://github.com/tomtomtommi/liteanystereo/blob/main/core/utils/utils.py"
click node_root_utils "https://github.com/tomtomtommi/liteanystereo/blob/main/Utils.py"
click node_assets "https://github.com/tomtomtommi/liteanystereo/tree/main/assets"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_demo,node_eval_stereo,node_eval_sh,node_flops,node_speed toneBlue
class node_models,node_las1,node_las2,node_las2h,node_fnet,node_agg,node_agg_fast,node_submodule toneAmber
class node_datasets,node_frame_utils,node_utils,node_root_utils toneMint
class node_checkpoints,node_datasets_ext,node_assets toneRose
```

## Components

### Entry points

Command-line scripts that run inference, evaluation, profiling, or complexity measurement.

- [demo.py](https://github.com/tomtomtommi/liteanystereo/blob/main/demo.py): CLI demo · Single stereo-pair inference and visualization entry point. · `demo.py`
- [evaluate_stereo.py](https://github.com/tomtomtommi/liteanystereo/blob/main/evaluate_stereo.py): CLI eval · Benchmark evaluation entry point for stereo datasets. · `evaluate_stereo.py`
- [evaluate.sh](https://github.com/tomtomtommi/liteanystereo/blob/main/evaluate.sh): shell wrapper · Convenience wrapper for paper benchmark runs. · `evaluate.sh`
- [flops_count.py](https://github.com/tomtomtommi/liteanystereo/blob/main/flops_count.py): MACs tool · Measures model complexity such as MACs and parameter cost. · `flops_count.py`
- [profile_speed.py](https://github.com/tomtomtommi/liteanystereo/blob/main/profile_speed.py): runtime profiler · Measures inference speed and GPU runtime behavior. · `profile_speed.py`

### Core models

Stereo network implementations and the factory that selects them.

- [models.py](https://github.com/tomtomtommi/liteanystereo/blob/main/core/models.py): model factory · Chooses and constructs the requested LAS architecture from CLI arguments. · `core/models.py`
- [LAS1](https://github.com/tomtomtommi/liteanystereo/blob/main/core/liteanystereo.py): stereo model · First-generation Lite Any Stereo network. · `core/liteanystereo.py`
- [LAS2](https://github.com/tomtomtommi/liteanystereo/blob/main/core/liteanystereov2.py): stereo model · Second-generation Lite Any Stereo network with size variants. · `core/liteanystereov2.py`
- [LAS2-H](https://github.com/tomtomtommi/liteanystereo/blob/main/core/liteanystereov2_H.py): stereo model · Specialized high-capacity LAS2 variant implementation. · `core/liteanystereov2_H.py`
- [fnet.py](https://github.com/tomtomtommi/liteanystereo/blob/main/core/fnet.py): feature net · Feature extraction backbone used by stereo models. · `core/fnet.py`
- [aggregation.py](https://github.com/tomtomtommi/liteanystereo/blob/main/core/aggregation.py): aggregation · Disparity cost aggregation and refinement logic. · `core/aggregation.py`
- [aggregation_fasternet.py](https://github.com/tomtomtommi/liteanystereo/blob/main/core/aggregation_fasternet.py): fast aggregation · Alternative aggregation path tuned for a faster backbone. · `core/aggregation_fasternet.py`
- [submodule.py](https://github.com/tomtomtommi/liteanystereo/blob/main/core/submodule.py): shared blocks · Reusable layers and helper modules shared across LAS variants. · `core/submodule.py`

### Support code

Dataset loading and shared utility helpers used by the runtime scripts and models.

- [stereo_datasets.py](https://github.com/tomtomtommi/liteanystereo/blob/main/core/stereo_datasets.py): dataset loaders · Benchmark dataset boundary for stereo evaluation. · `core/stereo_datasets.py`
- [frame_utils.py](https://github.com/tomtomtommi/liteanystereo/blob/main/core/utils/frame_utils.py): image I/O · Reads stereo images, disparity files, and related formats. · `core/utils/frame_utils.py`
- [utils.py](https://github.com/tomtomtommi/liteanystereo/blob/main/core/utils/utils.py): helpers · Shared math, padding, normalization, and evaluation helpers. · `core/utils/utils.py`
- [Utils.py](https://github.com/tomtomtommi/liteanystereo/blob/main/Utils.py): top-level helpers · Root-level helper module used by scripts and runtime utilities. · `Utils.py`

### External inputs

Dependencies that live outside the repository boundary, such as checkpoints and benchmark data.

- **checkpoints**: weights · External pretrained model weights loaded at runtime.
- **benchmarks**: datasets · External stereo benchmarks such as Middlebury, ETH3D, KITTI, and DrivingStereo.
- [assets](https://github.com/tomtomtommi/liteanystereo/tree/main/assets): examples · Example stereo pairs and demo artifacts bundled with the repo. · `assets`

## Connections

- demo.py → models.py: build model · The demo script constructs a selected LAS model before running inference.
- demo.py → frame_utils.py: load input · Demo input images and related files are read through frame utilities.
- demo.py → utils.py: pre/postprocess · Common helper routines support preprocessing and output handling.
- demo.py → assets: sample pairs · The demo can use bundled example stereo pairs.
- demo.py → checkpoints: load weights · Inference requires pretrained checkpoints.
- evaluate_stereo.py → models.py: build model · Evaluation scripts instantiate the requested LAS variant through the factory.
- evaluate_stereo.py → stereo_datasets.py: load benchmark · Evaluation builds dataset loaders for benchmark stereo sets.
- evaluate_stereo.py → benchmarks: evaluate on · Benchmark datasets are external to the repository.
- evaluate_stereo.py → checkpoints: load weights · Benchmark evaluation depends on pretrained weights.
- evaluate.sh → evaluate_stereo.py: wraps · Shell script orchestrates standard evaluation runs.
- flops_count.py → models.py: instantiate · Complexity measurement reuses the same model factory.
- profile_speed.py → models.py: instantiate · Profiling runs the selected model architecture.
- models.py → LAS1: selects · Factory can build LAS1.
- models.py → LAS2: selects · Factory can build LAS2 and its size variants.
- models.py → LAS2-H: selects · Factory can build the high-capacity LAS2 variant.
- LAS1 → fnet.py: uses · LAS1 relies on feature extraction blocks.
- LAS1 → aggregation.py: uses · LAS1 performs matching and aggregation.
- LAS1 → submodule.py: shares blocks · LAS1 uses shared layers and helper modules.
- LAS2 → fnet.py: uses · LAS2 relies on feature extraction blocks.
- LAS2 → aggregation_fasternet.py: uses · LAS2 can use the faster aggregation path.
- LAS2 → submodule.py: shares blocks · LAS2 uses shared layers and helper modules.
- LAS2-H → fnet.py: uses · LAS2-H relies on the shared feature network.
- LAS2-H → aggregation.py: uses · LAS2-H uses aggregation and refinement components.
- LAS2-H → submodule.py: shares blocks · LAS2-H uses shared layers and helper modules.
- stereo_datasets.py → frame_utils.py: reads data · Dataset loaders depend on file-format readers and parsers.
- stereo_datasets.py → utils.py: helpers · Dataset code uses shared utility routines.
- frame_utils.py → Utils.py: supports · Low-level I/O helpers are complemented by top-level utilities.