# nasa-impact/hls-foundation-os architecture

> Architecture diagram of the GitHub repository nasa-impact/hls-foundation-os, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-05-17). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/nasa-impact/hls-foundation-os
- Repository: https://github.com/nasa-impact/hls-foundation-os

## Overview

##### Project type and purpose
- This is a Python‐based full‐stack research tool for finetuning a ViT‐based masked autoencoder (“Prithvi”) on remote‐sensing segmentation tasks.
- It builds on the MMSegmentation framework, adds geospatial data pipelines, and exposes CLI scripts for training, testing, and inference.

##### File structure
- Top‐level modules:
- configs/ – three example config files for flood, burn‐scar and crop tasks
- data_splits/ – split definitions for each dataset
- geospatial_fm/ – Python package with dataset readers, pipelines, encoder‐decoder, and model definitions
- model_inference.py – CLI for running inference on GeoTIFF data
- exploration.ipynb – tutorial notebook
- setup.py – installs the package and dependencies
- External dependencies invoked by setup.py and mim (mmsegmentation, mmcv, torch)

##### README for insights
- The architecture: a ViT‐MAE encoder + neck (token→feature map) + segmentation head
- Data pipelines: custom MMSegmentation augmentations and channel‐format converters
- Training/testing orchestrated by `mim train/test mmsegmentation configs/*.py`

##### Main components and their interactions
a. Data Sources (external)
– Sen1Floods11, NASA HLS burn scars, HLS crop classification
– Downloaded via git‐lfs or Hugging Face, stored on disk
b. Data Splits & Preprocessing
– data_splits/<task>/…txt
– geospatial_fm/geospatial_pipelines.py defines MMSeg data pipelines (ToTensor, TorchPermute, TorchNormalize, augmentations)
c. Model Package
– geospatial_fm/geospatial_fm.py: builds backbone+neck+head architecture
– geospatial_fm/temporal_encoder_decoder.py: handles temporal sequences
– geospatial_fm/datasets.py: dataset classes for .tiff readers
d. Training & Testing
– configs/*.py: define model, dataset, optimizer, scheduler, runtime
– `mim train mmsegmentation configs/<task>.py` invokes MMSegmentation’s Trainer
– `mim test … --eval mIoU` evaluates metrics
e. Inference
– model_inference.py loads a config and checkpoint, reads GeoTIFFs, runs forward pass, writes segmentation masks
f. External Frameworks
– MMSegmentation, MMCV, PyTorch, OpenMIM

##### Architectural patterns & design principles
- Plugin architecture: custom dataset/pipeline components registered into MMSeg
- Modular separation:
- Data ingestion & augmentation
- Model definition (backbone+neck+head)
- Training orchestration via external CLI (mim)
- Inference as standalone script
- Conventions: channel‐last for pipelines, switch to channel‐first before model

## Diagram (Mermaid)

```mermaid
flowchart TD
    %% External Data Sources
    ED["External Data Sources"]:::cloud

    %% Data Storage
    DSp(("Data Splits & Storage")):::data
    click DSp "https://github.com/nasa-impact/hls-foundation-os/tree/main/data_splits/"

    %% Preprocessing Pipelines
    Pip["Preprocessing Pipelines"]:::data
    click Pip "https://github.com/nasa-impact/hls-foundation-os/blob/main/geospatial_fm/geospatial_pipelines.py"

    %% Dataset Readers
    DR["Dataset Readers"]:::core
    click DR "https://github.com/nasa-impact/hls-foundation-os/blob/main/geospatial_fm/datasets.py"

    %% Core Model Definition
    CM["Core Model Definition"]:::core
    click CM "https://github.com/nasa-impact/hls-foundation-os/blob/main/geospatial_fm/geospatial_fm.py"

    %% Temporal Encoder-Decoder
    TE["Temporal Encoder-Decoder Module"]:::core
    click TE "https://github.com/nasa-impact/hls-foundation-os/blob/main/geospatial_fm/temporal_encoder_decoder.py"

    %% Configuration Files
    subgraph "Training & Testing Configurations"
        CS1["burn_scars.py"]:::config
        CS2["multi_temporal_crop_classification.py"]:::config
        CS3["sen1floods11_config.py"]:::config
    end
    click CS1 "https://github.com/nasa-impact/hls-foundation-os/blob/main/configs/burn_scars.py"
    click CS2 "https://github.com/nasa-impact/hls-foundation-os/blob/main/configs/multi_temporal_crop_classification.py"
    click CS3 "https://github.com/nasa-impact/hls-foundation-os/blob/main/configs/sen1floods11_config.py"

    %% External Frameworks
    subgraph "External Frameworks"
        FW1["MMSegmentation"]:::ext
        FW2["MMCV"]:::ext
        FW3["PyTorch"]:::ext
        FW4["OpenMIM"]:::ext
    end

    %% Training & Testing Engine
    TEg["Training & Testing Engine"]:::externalFW

    %% Inference CLI Script
    Inf["Inference CLI Script"]:::config
    click Inf "https://github.com/nasa-impact/hls-foundation-os/blob/main/model_inference.py"

    %% Outputs
    Out(("Segmentation Masks")):::data

    %% Data & Control Flow
    ED -->|raw TIFFs| DSp
    DSp -->|file lists| DR
    DR -->|tensors| Pip
    Pip -->|augmented data| CM
    Pip -->|augmented data| TE
    CM -->|model inputs| TEg
    TE -->|encoded sequences| TEg
    CS1 --> TEg
    CS2 --> TEg
    CS3 --> TEg
    FW1 --> TEg
    FW2 --> TEg
    FW3 --> TEg
    FW4 --> TEg
    TEg -->|trained weights| Inf
    Inf -->|predicted masks| Out

    %% Legend
    subgraph Legend
        L1["Cylinder = Data Storage & Outputs"]:::data
        L2["Rectangle = Code Modules"]:::core
        L3["Parallelogram = Scripts & Configs"]:::config
        L4["Cloud = External Systems"]:::cloud
        L5["Dashed Cloud = External Frameworks"]:::ext
    end

    %% Styles
    classDef data fill:#ADD8E6,stroke:#000,stroke-width:1px;
    classDef core fill:#90EE90,stroke:#000,stroke-width:1px;
    classDef externalFW fill:#FFA500,stroke:#000,stroke-width:1px;
    classDef config fill:#DDA0DD,stroke:#000,stroke-width:1px;
    classDef cloud fill:#F0F8FF,stroke:#000,stroke-width:1px;
    classDef ext fill:#FFA500,stroke:#000,stroke-width:1px,stroke-dasharray: 2 2;
    class TEg externalFW
    class Inf config
    class CS1,CS2,CS3 config
    class DR,CM,TE core
    class FW1,FW2,FW3,FW4 ext
    class ED cloud
    class DSp,Out data
    class Pip data
```
