How muhkartal/slamai-istanbulcanyon works
Project type and purpose
• This is a domain‐specific research and development pipeline rather than a traditional web or mobile full‐stack app.
• Purpose: simulate drone flights in İstanbul‐style urban canyons (using AirSim), collect and preprocess data, run baseline SLAM, train and apply ML enhancements to SLAM (feature enhancement, loop closure, patch descriptors), integrate ML models with SLAM, and evaluate results.
File structure
• Top‐level Python scripts:
• airsim_setup.py – configures and launches the AirSim environment
• data_preprocessing.py – ingests raw sensor data, converts to SLAM/ML formats
• baseline_slam.py – wrappers/invocations for ORB-SLAM3, DSO, SVO
• ml_enhancement.py – training/inference of CNNs for feature, loop‐closure, descriptor networks
• slam_integration.py – stitches ML outputs into SLAM pipelines
• main.py – orchestrator for modes: simulate, preprocess, train_ml, evaluate_baseline, enhance_slam, evaluate_enhanced
• Secondary modules (from README’s structure):
• configs/ – YAML or JSON configuration files (paths, parameters)
• environments/ – definitions of simulation scenarios (urban_canyon, lighting, textureless)
• data/ – raw and converted datasets with ground‐truth
• models/ – saved ML models (ONNX, PyTorch)
• evaluation/ – scripts and metric calculators (ATE, RPE, success rate)
• results/ – processed outputs, plots, tables
• images/ – architecture and scenario illustrations
README for additional insights
• Describes a modular architecture allowing plug‐and‐play SLAM algorithms and ML modules, and a comprehensive evaluation framework.
• Lists required packages, external SLAM dependencies, and execution modes.
a. Components
1. AirSim Simulation
– Simulated drone in İstanbul urban canyon
– Outputs: raw RGB, depth, IMU, ground‐truth poses
2. Data Preprocessing
– Converts raw outputs into TUM or KITTI formats
– Normalizes, segments, augments datasets
3. Baseline SLAM Module
– ORB-SLAM3, DSO, SVO wrappers
– Consumes preprocessed data, writes baseline trajectories
4. ML Enhancement Module
– Feature Enhancement Network (CNN)
– Loop Closure Detection Network
– Patch Descriptor Network
– Serialization of trained models
5. SLAM Integration Layer
– Injects ML predictions into SLAM pipelines
– Modifies feature detection, matching, loop closing
6. Evaluation Framework
– Computes ATE, RPE, tracking success
– Generates plots and metrics tables
7. Configuration & Storage
– configs/ for parameterization
– models/, data/, results/ for persistence
8. External SLAM Libraries
– ORB-SLAM3, DSO, SVO (C++ compiled code)
b. Interactions and data flow
• AirSim → raw data → Data Preprocessing
• Preprocessed data → Baseline SLAM → baseline outputs
• Preprocessed data → ML Training → saved models
• Preprocessed data + saved models → SLAM Integration → enhanced SLAM outputs
• Baseline & enhanced outputs → Evaluation → results
• main.py orchestrates these flows based on command‐line mode flags
c. Architectural patterns and principles
• Pipeline pattern: clear sequential stages (simulate → preprocess → train → integrate → evaluate)
• Modular “plugin” approach for SLAM algorithms and ML networks
• Separation of concerns: each .py handles one responsibility
• Config‐driven design: parameters in configs/ allow scenario swaps without code changes
• External dependency isolation: C++ SLAM libraries invoked through wrappers
d. Key technologies and frameworks
• AirSim (drone simulator)
• Python 3.7+ with NumPy, OpenCV, PyTorch, ONNX Runtime
• C++17 SLAM libraries (ORB-SLAM3, DSO, SVO)
• Metrics with SciPy, pandas, matplotlib
• CLI orchestration via argparse in main.py