How isl-org/midas works
Project type and purpose
• This is MiDaS: a modular, multi-platform monocular depth-estimation toolkit.
• Core is a Python/PyTorch library (midas/…), with:
• A CLI (run.py) for inference on images or camera feed
• Conversion tools (tf/…) to export to TensorFlow and ONNX
• Multi-platform deployments: Docker, PyTorch Hub, mobile (Android & iOS), ROS1
File structure
• Top-level:
• midas/ – core model definitions (backbones, blocks, loader, transforms)
• run.py + utils.py – CLI entrypoint and helper functions
• tf/ – scripts to generate ONNX & TensorFlow models
• mobile/ – Android Studio project and Xcode project for iOS
• ros/ – ROS package with C++ node (midas_cpp/) and Python scripts
• Dockerfile + environment.yaml – containerization and environment setup
• weights/, input/, output/ – default folders for models, images, results
• Patterns:
• Core library with model‐API separation (model loader ↔ network definitions)
• Language & platform bindings layered on top of core
• Build/deploy scripts for each target
Key insights from README
• Purpose: compute relative depth maps from RGB inputs
• Supports multiple transformer & CNN backbones trained on varied datasets
• Offers several interfaces: CLI, Docker, PyTorch Hub, TF/ONNX, Android, iOS, ROS
Main components
a. Core inference engine
• midas package: backbones, blocks, base_model, model_loader, transforms
• PyTorch runtime
b. CLI & batch processing
• run.py orchestrates loading weights → preprocessing → inference → postprocessing
• input/ → output/ folders
c. Conversion tools
• tf/ scripts → produce .onnx, .pb models
d. Docker container
• Dockerfile + environment.yaml → isolated GPU-enabled environment
e. PyTorch Hub integration
• hubconf.py exposes models for torch.hub.load()
f. Mobile deployments
• Android:
– mobile/android/app (frontend UI + camera)
– lib_support, lib_task_api, models (TFLite integration, classification examples)
• iOS:
– mobile/ios/Midas (Swift code, storyboard, TFLite wrapper, camera feed manager)
– Podfile, RunScripts for model download
g. ROS1 node
• ros/midas_cpp (C++ executable, Python scripts for talker/listener)
• launch scripts, package.xml, CMakeLists.txt
Relationships and interactions
• Core engine feeds all other interfaces: CLI, Docker, Hub, TF/ONNX, mobile, ROS
• Conversion tools depend on core weights & network definitions
• Mobile & ROS components consume exported TFLite / ONNX / TorchScript models
• Docker bundles core engine + conversion scripts into one container
• PyTorch Hub directly calls core inference API (no conversion needed)
Architectural patterns & design principles
• Layered architecture:
1. Model definition layer (midas/)
2. Inference orchestration layer (run.py, TF/ONNX scripts)
3. Deployment layer (Docker, Hub, mobile, ROS)
• Separation of concerns: core model vs. platform-specific wrappers
• Single source of truth for model weights
• Reusable preprocessing/postprocessing transforms