# sher110106/moon_prototype architecture

> Architecture diagram of the GitHub repository sher110106/moon_prototype, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-07-05). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/sher110106/moon_prototype
- Repository: https://github.com/sher110106/moon_prototype

## Overview

##### Project Type and Purpose
- This is a geo-spatial data-processing and lightweight ML prototype (“Lunar Landslide Detection Prototype”) implemented as a multi-step pipeline.
- It ingests lunar imagery via ISRO’s Pradan CLI, applies rule-based terrain analysis and ML (U-Net + YOLOv8), and outputs validated landslide/boulder detections in GeoPackage + metrics CSV.

##### File Structure
- Top-level orchestration:
- run_prototype.py – CLI entry point for running individual or all steps
- config.yaml – global parameters (e.g., AOI path, thresholds)
- scripts/00_env_setup – Conda environment + GDAL configuration snippets
- Processing pipeline (scripts/01–11):
- 01_data_acquisition.sh – downloads data via pradan-cli
- 02_raster_preprocessing.py – COG conversion, mosaicking, clipping
- 03_coregistration.sh – manual GCP collection in QGIS + co-registration
- 04_hapke_normalisation.py – photometric correction (Hapke model)
- 05_terrain_derivatives.py – slope & curvature using RichDEM
- 06_rule_based_baseline.py – thresholded slope/curvature rule engine
- 07_annotation_sprint.md – manual LabelMe annotation sprint
- 08_light_ml_models.py – U-Net for segmentation & YOLOv8 validation
- 09_fusion_and_filter.py – combine rule-based + ML outputs & filter
- 10_metrics_audit.py – compute IoU, AP50, runtime metrics
- 11_visuals_packaging.py – generate maps & package outputs
- Other directories:
- data/ – input AOI geojson + any rasters or shapefiles
- outputs/ – final GPKG & metrics CSV
- notebooks/ – Jupyter analysis & Prototype_Report.ipynb
- reports/ – PROTOTYPE_WORKFLOW.md with detailed documentation
- tests (test_improvements.py) – ad hoc testing utilities

##### README Insights
- Full pipeline automation via `python run_prototype.py --all`
- Manual steps (☑) at steps 0 (env), 3 (co-registration), 7 (annotation)
- Key algorithms & tech: GDAL 3.8+, RichDEM, U-Net (ResNet18), YOLOv8n, LabelMe, QGIS

##### Main Components
a. Environment Setup
- Conda environment (scripts/00_env_setup)
- GDAL & QGIS configurations
b. Data Ingestion
- External Data Sources: ISRO Pradan data portal (via pradan-cli)
- Script: 01_data_acquisition.sh
- Input: raw TMC Ortho, DTM, OHRC
c. Preprocessing & Coregistration
- 02_raster_preprocessing.py → COG, reproject, clip
- 03_coregistration.sh → manual GCP collection in QGIS
d. Photometric & Terrain Processing
- 04_hapke_normalisation.py → cosine + Hapke correction
- 05_terrain_derivatives.py → compute slope/curvature
e. Detection Engines
- Rule-Based: 06_rule_based_baseline.py
- ML Pipeline:
- Manual annotation (07_annotation_sprint.md via LabelMe)
- ML training & inference (08_light_ml_models.py with U-Net & YOLOv8)
f. Fusion, Metrics & Packaging
- 09_fusion_and_filter.py → merge rule + ML outputs
- 10_metrics_audit.py → IoU/AP metrics, runtime profiling
- 11_visuals_packaging.py → map figures, GeoPackage & CSV

##### Relationships & Data Flow
- Orchestrator: run_prototype.py invokes each numbered script in sequence
- Data download → preprocessing → co-registration → normalization → derivatives
- Derivatives feed simultaneously into rule-based & ML branches
- Both branches converge at fusion & filtering
- Fusion output → metrics audit → visuals packaging → final outputs

##### Architectural Patterns & Design Principles
- Linear, staged pipeline (ETL style) with manual intervention points
- Modular scripts (each step is independently executable via CLI flags)
- Separation of concerns: data ingestion, transformation, detection, evaluation
- Extensibility: new scripts can slot into pipeline (scripts/XX_new_step.py)

##### Key Technologies & Frameworks
- Python 3.10 for all processing scripts
- Bash for shell orchestration (data acquisition, coregistration)
- Conda environment management (environment.yml)
- GDAL & RichDEM for geospatial operations
- U-Net (segmentation) & YOLOv8n (object detection) for ML
- QGIS & LabelMe for manual annotation

##### Elements
- Clear labels for scripts (e.g., “02_raster_preprocessing.py”)
- Data artifacts at each stage (e.g., slope.tif, curvature.tif)
- External services (Pradan data portal, QGIS for manual GCP)
- Orchestration (run_prototype.py) at top or central hub
- Blue = automated steps
- Yellow = manual intervention
- Green = data repositories

##### Final Advice
- Manual vs. automated to surface human dependencies

## Diagram (Mermaid)

```mermaid
flowchart TD
    %% Orchestration Layer
    subgraph "Orchestration"
        Run["run_prototype.py<br/>(Orchestrator)"]:::orches
        Config["config.yaml<br/>(Global Config)"]:::config
    end

    %% Environment Setup Layer
    subgraph "1. Environment Setup"
        EnvYML["environment.yml"]:::shell
        GDAL_SNIP["gdal_bashrc_snippet.sh"]:::shell
    end

    %% Data Ingestion Layer
    subgraph "2. Data Ingestion"
        Pradan["01_data_acquisition.sh<br/>(Pradan CLI)"]:::shell
        ExtData["ISRO Pradan Data Portal"]:::external
        AOI["data/aoi.geojson"]:::data
    end

    %% Preprocessing & Coregistration Layer
    subgraph "3. Preprocessing & Coregistration"
        Preproc["02_raster_preprocessing.py<br/>(COG & Clip)"]:::python
        Coreg["03_coregistration.sh<br/>(Manual GCP in QGIS)"]:::manual
    end

    %% Photometric & Terrain Processing Layer
    subgraph "4. Photometric & Terrain"
        Hapke["04_hapke_normalisation.py<br/>(Hapke Correction)"]:::python
        Deriv["05_terrain_derivatives.py<br/>(Slope & Curvature)"]:::python
    end

    %% Detection Engines Layer
    subgraph "5. Detection Engines"
        Rule["06_rule_based_baseline.py<br/>(Rule-Based)"]:::python
        Annot["07_annotation_sprint.md<br/>(LabelMe)"]:::manual
        ML["08_light_ml_models.py<br/>(U-Net & YOLOv8)"]:::python
    end

    %% Fusion, Metrics & Packaging Layer
    subgraph "6. Fusion, Metrics & Packaging"
        Fuse["09_fusion_and_filter.py<br/>(Merge & Filter)"]:::python
        Metrics["10_metrics_audit.py<br/>(IoU, AP50)"]:::python
        Visuals["11_visuals_packaging.py<br/>(Maps & GPKG/CSV)"]:::python
        Outputs["outputs/prototype_metrics.csv<br/> & GeoPackage"]:::data
    end

    %% Jupyter & Documentation & Tests
    subgraph "Support Artifacts"
        ReportNB["Prototype_Report.ipynb"]:::notebook
        WorkflowDoc["reports/PROTOTYPE_WORKFLOW.md"]:::doc
        TestUtil["test_improvements.py"]:::test
    end

    %% Data Flow
    Run -->|uses| Config
    Run --> EnvYML
    Run --> Pradan
    Pradan --> AOI
    Pradan --> Preproc
    Preproc --> Coreg
    Coreg --> Hapke
    Hapke --> Deriv
    Deriv --> Rule
    Deriv --> ML
    Rule --> Fuse
    ML --> Fuse
    Fuse --> Metrics
    Metrics --> Visuals
    Visuals --> Outputs

    %% External Systems
    Pradan --> ExtData
    Coreg -->|QGIS Manual Step| Annot

    %% Click Events
    click EnvYML "https://github.com/sher110106/moon_prototype/blob/main/scripts/00_env_setup/environment.yml"
    click GDAL_SNIP "https://github.com/sher110106/moon_prototype/blob/main/scripts/00_env_setup/gdal_bashrc_snippet.sh"
    click Pradan "https://github.com/sher110106/moon_prototype/blob/main/scripts/01_data_acquisition.sh"
    click Preproc "https://github.com/sher110106/moon_prototype/blob/main/scripts/02_raster_preprocessing.py"
    click Coreg "https://github.com/sher110106/moon_prototype/blob/main/scripts/03_coregistration.sh"
    click Hapke "https://github.com/sher110106/moon_prototype/blob/main/scripts/04_hapke_normalisation.py"
    click Deriv "https://github.com/sher110106/moon_prototype/blob/main/scripts/05_terrain_derivatives.py"
    click Rule "https://github.com/sher110106/moon_prototype/blob/main/scripts/06_rule_based_baseline.py"
    click Annot "https://github.com/sher110106/moon_prototype/blob/main/scripts/07_annotation_sprint.md"
    click ML "https://github.com/sher110106/moon_prototype/blob/main/scripts/08_light_ml_models.py"
    click Fuse "https://github.com/sher110106/moon_prototype/blob/main/scripts/09_fusion_and_filter.py"
    click Metrics "https://github.com/sher110106/moon_prototype/blob/main/scripts/10_metrics_audit.py"
    click Visuals "https://github.com/sher110106/moon_prototype/blob/main/scripts/11_visuals_packaging.py"
    click Run "https://github.com/sher110106/moon_prototype/blob/main/run_prototype.py"
    click Config "https://github.com/sher110106/moon_prototype/blob/main/config.yaml"

    %% Styles
    classDef orches    fill:#DDEEFF,stroke:#3399FF,stroke-width:2px
    classDef config    fill:#EEFFDD,stroke:#33AA33,stroke-width:2px
    classDef shell     fill:#E0F7FA,stroke:#006064,stroke-width:1px
    classDef python    fill:#E8EAF6,stroke:#3F51B5,stroke-width:1px
    classDef manual    fill:#FFF9C4,stroke:#FBC02D,stroke-width:1px
    classDef external  fill:#ECEFF1,stroke:#607D8B,stroke-width:1px
    classDef data      fill:#E8F5E9,stroke:#2E7D32,stroke-width:1px
    classDef notebook  fill:#F3E5F5,stroke:#8E24AA,stroke-width:1px
    classDef doc       fill:#FFEBEE,stroke:#C62828,stroke-width:1px
    classDef test      fill:#FFF3E0,stroke:#EF6C00,stroke-width:1px
```
