How tomtomtommi/liteanystereo works
1. Repository purpose
This is an inference/evaluation codebase for the Lite Any Stereo family, not a training-first project. The main architectural concern is selecting a LAS variant, loading a checkpoint, running stereo matching, and reporting either visual output, benchmark metrics, MACs, or runtime.
2. Top-level execution entry points
The repo is organized around a small set of scripts:
- demo.py for single stereo-pair inference and visualization
- evaluate_stereo.py for benchmark evaluation on standard stereo datasets
- evaluate.sh as a convenience wrapper for the paper’s benchmark runs
- flops_count.py for model complexity reporting
- profile_speed.py for runtime measurement
These scripts are the main “application layer” and all delegate into the core/ package.
3. Model family boundary
The core/ package contains the actual stereo networks. There are two model generations:
- LAS1 in core/liteanystereo.py
- LAS2 in core/liteanystereov2.py, with size variants S/M/L/H
- A separate core/liteanystereov2_H.py suggests a specialized high-capacity variant implementation
The README indicates LAS2 defaults to the M model unless --model_size is specified.
4. Shared model building blocks
Several files indicate reusable neural-network subsystems:
- core/fnet.py likely implements the feature extraction network
- core/aggregation.py and core/aggregation_fasternet.py likely implement disparity cost aggregation / refinement
- core/submodule.py likely holds shared layers, blocks, and helper modules used across LAS1/LAS2
- core/models.py likely acts as the model factory or top-level wrapper that chooses the right architecture from CLI arguments
Architecturally, these form the internal decomposition of “feature extraction → matching/aggregation → disparity regression/output”.
5. Inference data flow
The primary runtime path is stereo pair input → preprocessing → model forward pass → disparity output.
In the demo path, the outputs are written as:
- a disparity visualization
- a raw disparity array
- optional point-cloud artifacts
This means the repo’s output boundary is not only numeric prediction but also visualization/export for downstream use.
6. Dataset and evaluation subsystem
core/stereo_datasets.py is the dataset boundary for benchmark evaluation. The README shows support for:
- middlebury_F
- middlebury_H
- middlebury_Q
- eth3d
- kitti
- drivingstereo
evaluate_stereo.py likely builds the chosen dataset loader, runs inference, and computes benchmark-specific metrics. This is the main path for reproducing paper results.
7. Utility layer
core/utils/ and the root-level Utils.py provide support code around image/frame loading and general helpers:
- core/utils/frame_utils.py likely handles reading stereo images, disparity maps, and related file formats
- core/utils/utils.py likely contains common math, padding, normalization, logging, or evaluation helpers
These utilities sit below the model and dataset layers and keep the entry-point scripts small.
8. Assets and external artifacts
assets/ contains example stereo image pairs and a performance figure used by the demo and README.
Checkpoints are explicitly external, stored under ./checkpoints/ and fetched from Hugging Face or Google Drive. This means model weights are not part of the code path itself but are a required runtime dependency.
9. Measurement and profiling tools
Two scripts support non-functional analysis:
- flops_count.py measures MACs / model complexity
- profile_speed.py measures inference runtime, including CUDA synchronization on GPU
These scripts likely instantiate the same model factory as demo/evaluation, but stop at measurement rather than producing disparities.
10. Technology/runtime boundaries
The repo is centered on PyTorch-style model inference for stereo vision, with command-line Python entry points and shell-based evaluation orchestration. The external boundaries that matter architecturally are:
- pretrained checkpoints
- benchmark datasets
- GPU execution for runtime profiling and practical deployment
There is no sign of a server, web UI, or training pipeline in this repository; its architecture is focused on offline inference, benchmarking, and profiling.
11. How to draw it
A good architecture diagram would show:
- CLI scripts at the top
- a shared core/ model factory beneath them
- model variants branching into LAS1 and LAS2
- shared feature/aggregation modules underneath the variants
- dataset loaders and utility I/O on the side feeding inputs into the model
- outputs flowing to visualization files, benchmark metrics, or profiling reports
- checkpoints and datasets as external dependencies outside the repo boundary
Main entry points: models.py models.py · demo.py demo.py · evaluate_stereo.py evaluate_stereo.py · liteanystereo.py LAS1
Components 20
Entry points
Command-line scripts that run inference, evaluation, profiling, or complexity measurement.
- demo.py · Single stereo-pair inference and visualization entry point.
demo.py - evaluate_stereo.py · Benchmark evaluation entry point for stereo datasets.
evaluate_stereo.py - evaluate.sh · Convenience wrapper for paper benchmark runs.
evaluate.sh - flops_count.py · Measures model complexity such as MACs and parameter cost.
flops_count.py - profile_speed.py · Measures inference speed and GPU runtime behavior.
profile_speed.py
Core models
Stereo network implementations and the factory that selects them.
- models.py · Chooses and constructs the requested LAS architecture from CLI arguments.
core/models.py - LAS1 · First-generation Lite Any Stereo network.
core/liteanystereo.py - LAS2 · Second-generation Lite Any Stereo network with size variants.
core/liteanystereov2.py - LAS2-H · Specialized high-capacity LAS2 variant implementation.
core/liteanystereov2_H.py - fnet.py · Feature extraction backbone used by stereo models.
core/fnet.py - aggregation.py · Disparity cost aggregation and refinement logic.
core/aggregation.py - aggregation_fasternet.py · Alternative aggregation path tuned for a faster backbone.
core/aggregation_fasternet.py - submodule.py · Reusable layers and helper modules shared across LAS variants.
core/submodule.py
Support code
Dataset loading and shared utility helpers used by the runtime scripts and models.
- stereo_datasets.py · Benchmark dataset boundary for stereo evaluation.
core/stereo_datasets.py - frame_utils.py · Reads stereo images, disparity files, and related formats.
core/utils/frame_utils.py - utils.py · Shared math, padding, normalization, and evaluation helpers.
core/utils/utils.py - Utils.py · Root-level helper module used by scripts and runtime utilities.
Utils.py
External inputs
Dependencies that live outside the repository boundary, such as checkpoints and benchmark data.
- checkpoints · External pretrained model weights loaded at runtime.
- benchmarks · External stereo benchmarks such as Middlebury, ETH3D, KITTI, and DrivingStereo.
- assets · Example stereo pairs and demo artifacts bundled with the repo.
assets
Connections 27
- demo.py → models.py: build model · The demo script constructs a selected LAS model before running inference.
- demo.py → frame_utils.py: load input · Demo input images and related files are read through frame utilities.
- demo.py → utils.py: pre/postprocess · Common helper routines support preprocessing and output handling.
- demo.py → assets: sample pairs · The demo can use bundled example stereo pairs.
- demo.py → checkpoints: load weights · Inference requires pretrained checkpoints.
- evaluate_stereo.py → models.py: build model · Evaluation scripts instantiate the requested LAS variant through the factory.
- evaluate_stereo.py → stereo_datasets.py: load benchmark · Evaluation builds dataset loaders for benchmark stereo sets.
- evaluate_stereo.py → benchmarks: evaluate on · Benchmark datasets are external to the repository.
- evaluate_stereo.py → checkpoints: load weights · Benchmark evaluation depends on pretrained weights.
- evaluate.sh → evaluate_stereo.py: wraps · Shell script orchestrates standard evaluation runs.
- flops_count.py → models.py: instantiate · Complexity measurement reuses the same model factory.
- profile_speed.py → models.py: instantiate · Profiling runs the selected model architecture.
- models.py → LAS1: selects · Factory can build LAS1.
- models.py → LAS2: selects · Factory can build LAS2 and its size variants.
- models.py → LAS2-H: selects · Factory can build the high-capacity LAS2 variant.
- LAS1 → fnet.py: uses · LAS1 relies on feature extraction blocks.
- LAS1 → aggregation.py: uses · LAS1 performs matching and aggregation.
- LAS1 → submodule.py: shares blocks · LAS1 uses shared layers and helper modules.
- LAS2 → fnet.py: uses · LAS2 relies on feature extraction blocks.
- LAS2 → aggregation_fasternet.py: uses · LAS2 can use the faster aggregation path.
- LAS2 → submodule.py: shares blocks · LAS2 uses shared layers and helper modules.
- LAS2-H → fnet.py: uses · LAS2-H relies on the shared feature network.
- LAS2-H → aggregation.py: uses · LAS2-H uses aggregation and refinement components.
- LAS2-H → submodule.py: shares blocks · LAS2-H uses shared layers and helper modules.
- stereo_datasets.py → frame_utils.py: reads data · Dataset loaders depend on file-format readers and parsers.
- stereo_datasets.py → utils.py: helpers · Dataset code uses shared utility routines.
- frame_utils.py → Utils.py: supports · Low-level I/O helpers are complemented by top-level utilities.