# isl-org/midas architecture

> Architecture diagram of the GitHub repository isl-org/midas, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-05-21). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/isl-org/midas
- Repository: https://github.com/isl-org/midas

## Overview

##### Project type and purpose
- This is MiDaS: a modular, multi-platform monocular depth-estimation toolkit.
- Core is a Python/PyTorch library (`midas/…`), with:
- A CLI (`run.py`) for inference on images or camera feed
- Conversion tools (`tf/…`) to export to TensorFlow and ONNX
- Multi-platform deployments: Docker, PyTorch Hub, mobile (Android & iOS), ROS1

##### File structure
- Top-level:
- `midas/` – core model definitions (backbones, blocks, loader, transforms)
- `run.py` + `utils.py` – CLI entrypoint and helper functions
- `tf/` – scripts to generate ONNX & TensorFlow models
- `mobile/` – Android Studio project and Xcode project for iOS
- `ros/` – ROS package with C++ node (`midas_cpp/`) and Python scripts
- `Dockerfile` + `environment.yaml` – containerization and environment setup
- `weights/`, `input/`, `output/` – default folders for models, images, results
- Patterns:
- Core library with model‐API separation (model loader ↔ network definitions)
- Language & platform bindings layered on top of core
- Build/deploy scripts for each target

##### Key insights from README
- Purpose: compute relative depth maps from RGB inputs
- Supports multiple transformer & CNN backbones trained on varied datasets
- Offers several interfaces: CLI, Docker, PyTorch Hub, TF/ONNX, Android, iOS, ROS

##### Main components
a. Core inference engine
- midas package: backbones, blocks, base_model, model_loader, transforms
- PyTorch runtime
b. CLI & batch processing
- `run.py` orchestrates loading weights → preprocessing → inference → postprocessing
- `input/` → `output/` folders
c. Conversion tools
- `tf/` scripts → produce `.onnx`, `.pb` models
d. Docker container
- `Dockerfile` + `environment.yaml` → isolated GPU-enabled environment
e. PyTorch Hub integration
- `hubconf.py` exposes models for `torch.hub.load()`
f. Mobile deployments
- Android:
– `mobile/android/app` (frontend UI + camera)
– `lib_support`, `lib_task_api`, `models` (TFLite integration, classification examples)
- iOS:
– `mobile/ios/Midas` (Swift code, storyboard, TFLite wrapper, camera feed manager)
– `Podfile`, RunScripts for model download
g. ROS1 node
- `ros/midas_cpp` (C++ executable, Python scripts for talker/listener)
- launch scripts, package.xml, CMakeLists.txt

##### Relationships and interactions
- Core engine feeds all other interfaces: CLI, Docker, Hub, TF/ONNX, mobile, ROS
- Conversion tools depend on core weights & network definitions
- Mobile & ROS components consume exported TFLite / ONNX / TorchScript models
- Docker bundles core engine + conversion scripts into one container
- PyTorch Hub directly calls core inference API (no conversion needed)

##### Architectural patterns & design principles
- Layered architecture:
1. Model definition layer (`midas/`)
2. Inference orchestration layer (`run.py`, TF/ONNX scripts)
3. Deployment layer (Docker, Hub, mobile, ROS)
- Separation of concerns: core model vs. platform-specific wrappers
- Single source of truth for model weights
- Reusable preprocessing/postprocessing transforms

## Diagram (Mermaid)

```mermaid
flowchart TD
    %% Core Layer
    Core["Core Inference Engine<br/>(PyTorch + midas package)"]:::core
    click Core "https://github.com/isl-org/midas/tree/master/midas/"

    %% CLI & Batch Processing
    CLI["CLI & Batch Processing<br/>run.py, utils.py"]:::deployment
    click CLI "https://github.com/isl-org/midas/blob/master/run.py"
    click CLI "https://github.com/isl-org/midas/blob/master/utils.py"
    Input["Input Folder<br/>(input/)"]:::deployment
    click Input "https://github.com/isl-org/midas/tree/master/input/"
    Output["Output Folder<br/>(output/)"]:::deployment
    click Output "https://github.com/isl-org/midas/tree/master/output/"

    %% Conversion Tools
    subgraph "TF/ONNX Conversion Tools" 
        ONNX["make_onnx_model.py"]:::deployment
        click ONNX "https://github.com/isl-org/midas/blob/master/tf/make_onnx_model.py"
        runONNX["run_onnx.py"]:::deployment
        click runONNX "https://github.com/isl-org/midas/blob/master/tf/run_onnx.py"
        runPB["run_pb.py"]:::deployment
        click runPB "https://github.com/isl-org/midas/blob/master/tf/run_pb.py"
        transformsTF["transforms.py"]:::deployment
        click transformsTF "https://github.com/isl-org/midas/blob/master/tf/transforms.py"
        utilsTF["utils.py"]:::deployment
        click utilsTF "https://github.com/isl-org/midas/blob/master/tf/utils.py"
    end

    %% Docker Container
    subgraph "Docker Container"
        DockerfileNode["Dockerfile"]:::deployment
        click DockerfileNode "https://github.com/isl-org/midas/tree/master/Dockerfile"
        EnvFile["environment.yaml"]:::deployment
        click EnvFile "https://github.com/isl-org/midas/blob/master/environment.yaml"
    end

    %% PyTorch Hub
    Hub["PyTorch Hub Integration<br/>(hubconf.py)"]:::deployment
    click Hub "https://github.com/isl-org/midas/blob/master/hubconf.py"

    %% Mobile Android
    subgraph "Android Deployment"
        AndroidApp["Android App<br/>(mobile/android/app/)"]:::deployment
        click AndroidApp "https://github.com/isl-org/midas/tree/master/mobile/android/app/"
        LibSupport["lib_support"]:::deployment
        click LibSupport "https://github.com/isl-org/midas/tree/master/mobile/android/lib_support/"
        LibTask["lib_task_api"]:::deployment
        click LibTask "https://github.com/isl-org/midas/tree/master/mobile/android/lib_task_api/"
        AndroidModels["models"]:::deployment
        click AndroidModels "https://github.com/isl-org/midas/tree/master/mobile/android/models/"
    end

    %% Mobile iOS
    subgraph "iOS Deployment"
        iOSApp["iOS App<br/>(mobile/ios/Midas/)"]:::deployment
        click iOSApp "https://github.com/isl-org/midas/tree/master/mobile/ios/Midas/"
        Pod["Podfile"]:::deployment
        click Pod "https://github.com/isl-org/midas/tree/master/mobile/ios/Podfile"
        Script["download_models.sh"]:::deployment
        click Script "https://github.com/isl-org/midas/blob/master/mobile/ios/RunScripts/download_models.sh"
    end

    %% ROS1 Node
    subgraph "ROS1 Deployment"
        ROSNode["ROS1 Node<br/>(ros/midas_cpp/)"]:::deployment
        click ROSNode "https://github.com/isl-org/midas/tree/master/ros/midas_cpp/"
        Additions["Additions & Launch Scripts"]:::deployment
        click Additions "https://github.com/isl-org/midas/tree/master/ros/additions/"
        LaunchSh["launch_midas_cpp.sh"]:::deployment
        click LaunchSh "https://github.com/isl-org/midas/blob/master/ros/launch_midas_cpp.sh"
    end

    %% Weights Store
    Weights["Weights Store<br/>(weights/)"]:::deployment
    click Weights "https://github.com/isl-org/midas/tree/master/weights/"

    %% External Dependencies
    EnvDep["Conda/Docker Env<br/>NVIDIA GPU Drivers"]:::external
    PyTorchDep["PyTorch Runtime"]:::external
    OpenVINO["OpenVINO"]:::external
    TFLiteDep["TFLite Runtime"]:::external
    ROSDep["ROS1"]:::external

    %% Data Flows & Relationships
    Input -->|reads images| CLI
    CLI -->|invokes| Core
    Core -->|writes depth maps| Output

    Core --> ONNX
    Core --> runONNX
    Core --> runPB
    Core --> transformsTF
    Core --> utilsTF

    DockerfileNode -->|builds| DockerfileNode
    EnvFile -->|defines| DockerfileNode
    DockerfileNode -->|packages| Core
    DockerfileNode -->|packages| CLI

    Hub -->|calls API| Core

    Core -->|exports TFLite| AndroidModels
    AndroidModels --> LibSupport
    AndroidModels --> LibTask
    LibSupport --> AndroidApp
    LibTask --> AndroidApp
    TFLiteDep --> AndroidApp

    Core -->|exports TFLite| iOSApp
    Pod --> iOSApp
    Script --> iOSApp
    TFLiteDep --> iOSApp

    ROSNode -->|calls C++/Python API| Core
    LaunchSh --> ROSNode
    Additions --> ROSNode
    ROSDep --> ROSNode

    Weights -->|loaded by| Core

    EnvDep --> DockerfileNode
    PyTorchDep --> Core
    OpenVINO --> Core

    %% Styling
    classDef core fill:#cce5ff,stroke:#2b7bb9;
    classDef deployment fill:#d4edda,stroke:#256029;
    classDef external fill:#e2e3e5,stroke:#6c757d,stroke-dasharray: 5 5;
```
