How sher110106/moon_prototype works
Project Type and Purpose
• This is a geo-spatial data-processing and lightweight ML prototype (“Lunar Landslide Detection Prototype”) implemented as a multi-step pipeline.
• It ingests lunar imagery via ISRO’s Pradan CLI, applies rule-based terrain analysis and ML (U-Net + YOLOv8), and outputs validated landslide/boulder detections in GeoPackage + metrics CSV.
File Structure
• Top-level orchestration:
• run_prototype.py – CLI entry point for running individual or all steps
• config.yaml – global parameters (e.g., AOI path, thresholds)
• scripts/00_env_setup – Conda environment + GDAL configuration snippets
• Processing pipeline (scripts/01–11):
• 01_data_acquisition.sh – downloads data via pradan-cli
• 02_raster_preprocessing.py – COG conversion, mosaicking, clipping
• 03_coregistration.sh – manual GCP collection in QGIS + co-registration
• 04_hapke_normalisation.py – photometric correction (Hapke model)
• 05_terrain_derivatives.py – slope & curvature using RichDEM
• 06_rule_based_baseline.py – thresholded slope/curvature rule engine
• 07_annotation_sprint.md – manual LabelMe annotation sprint
• 08_light_ml_models.py – U-Net for segmentation & YOLOv8 validation
• 09_fusion_and_filter.py – combine rule-based + ML outputs & filter
• 10_metrics_audit.py – compute IoU, AP50, runtime metrics
• 11_visuals_packaging.py – generate maps & package outputs
• Other directories:
• data/ – input AOI geojson + any rasters or shapefiles
• outputs/ – final GPKG & metrics CSV
• notebooks/ – Jupyter analysis & Prototype_Report.ipynb
• reports/ – PROTOTYPE_WORKFLOW.md with detailed documentation
• tests (test_improvements.py) – ad hoc testing utilities
README Insights
• Full pipeline automation via python run_prototype.py --all
• Manual steps (☑) at steps 0 (env), 3 (co-registration), 7 (annotation)
• Key algorithms & tech: GDAL 3.8+, RichDEM, U-Net (ResNet18), YOLOv8n, LabelMe, QGIS
Main Components
a. Environment Setup
• Conda environment (scripts/00_env_setup)
• GDAL & QGIS configurations
b. Data Ingestion
• External Data Sources: ISRO Pradan data portal (via pradan-cli)
• Script: 01_data_acquisition.sh
• Input: raw TMC Ortho, DTM, OHRC
c. Preprocessing & Coregistration
• 02_raster_preprocessing.py → COG, reproject, clip
• 03_coregistration.sh → manual GCP collection in QGIS
d. Photometric & Terrain Processing
• 04_hapke_normalisation.py → cosine + Hapke correction
• 05_terrain_derivatives.py → compute slope/curvature
e. Detection Engines
• Rule-Based: 06_rule_based_baseline.py
• ML Pipeline:
• Manual annotation (07_annotation_sprint.md via LabelMe)
• ML training & inference (08_light_ml_models.py with U-Net & YOLOv8)
f. Fusion, Metrics & Packaging
• 09_fusion_and_filter.py → merge rule + ML outputs
• 10_metrics_audit.py → IoU/AP metrics, runtime profiling
• 11_visuals_packaging.py → map figures, GeoPackage & CSV
Relationships & Data Flow
• Orchestrator: run_prototype.py invokes each numbered script in sequence
• Data download → preprocessing → co-registration → normalization → derivatives
• Derivatives feed simultaneously into rule-based & ML branches
• Both branches converge at fusion & filtering
• Fusion output → metrics audit → visuals packaging → final outputs
Architectural Patterns & Design Principles
• Linear, staged pipeline (ETL style) with manual intervention points
• Modular scripts (each step is independently executable via CLI flags)
• Separation of concerns: data ingestion, transformation, detection, evaluation
• Extensibility: new scripts can slot into pipeline (scripts/XX_new_step.py)
Key Technologies & Frameworks
• Python 3.10 for all processing scripts
• Bash for shell orchestration (data acquisition, coregistration)
• Conda environment management (environment.yml)
• GDAL & RichDEM for geospatial operations
• U-Net (segmentation) & YOLOv8n (object detection) for ML
• QGIS & LabelMe for manual annotation
Elements
• Clear labels for scripts (e.g., “02_raster_preprocessing.py”)
• Data artifacts at each stage (e.g., slope.tif, curvature.tif)
• External services (Pradan data portal, QGIS for manual GCP)
• Orchestration (run_prototype.py) at top or central hub
• Blue = automated steps
• Yellow = manual intervention
• Green = data repositories
Final Advice
• Manual vs. automated to surface human dependencies