How nasa-impact/hls-foundation-os works
Project type and purpose
• This is a Python‐based full‐stack research tool for finetuning a ViT‐based masked autoencoder (“Prithvi”) on remote‐sensing segmentation tasks.
• It builds on the MMSegmentation framework, adds geospatial data pipelines, and exposes CLI scripts for training, testing, and inference.
File structure
• Top‐level modules:
• configs/ – three example config files for flood, burn‐scar and crop tasks
• data_splits/ – split definitions for each dataset
• geospatial_fm/ – Python package with dataset readers, pipelines, encoder‐decoder, and model definitions
• model_inference.py – CLI for running inference on GeoTIFF data
• exploration.ipynb – tutorial notebook
• setup.py – installs the package and dependencies
• External dependencies invoked by setup.py and mim (mmsegmentation, mmcv, torch)
README for insights
• The architecture: a ViT‐MAE encoder + neck (token→feature map) + segmentation head
• Data pipelines: custom MMSegmentation augmentations and channel‐format converters
• Training/testing orchestrated by mim train/test mmsegmentation configs/*.py
Main components and their interactions
a. Data Sources (external)
– Sen1Floods11, NASA HLS burn scars, HLS crop classification
– Downloaded via git‐lfs or Hugging Face, stored on disk
b. Data Splits & Preprocessing
– data_splits/<task>/…txt
– geospatial_fm/geospatial_pipelines.py defines MMSeg data pipelines (ToTensor, TorchPermute, TorchNormalize, augmentations)
c. Model Package
– geospatial_fm/geospatial_fm.py: builds backbone+neck+head architecture
– geospatial_fm/temporal_encoder_decoder.py: handles temporal sequences
– geospatial_fm/datasets.py: dataset classes for .tiff readers
d. Training & Testing
– configs/*.py: define model, dataset, optimizer, scheduler, runtime
– mim train mmsegmentation configs/<task>.py invokes MMSegmentation’s Trainer
– mim test … --eval mIoU evaluates metrics
e. Inference
– model_inference.py loads a config and checkpoint, reads GeoTIFFs, runs forward pass, writes segmentation masks
f. External Frameworks
– MMSegmentation, MMCV, PyTorch, OpenMIM
Architectural patterns & design principles
• Plugin architecture: custom dataset/pipeline components registered into MMSeg
• Modular separation:
• Data ingestion & augmentation
• Model definition (backbone+neck+head)
• Training orchestration via external CLI (mim)
• Inference as standalone script
• Conventions: channel‐last for pipelines, switch to channel‐first before model