How mbzuai-oryx/terrafm works
TerraFM is a pretrained multisensor Earth-observation vision model, exposed as standalone Base and Large PyTorch models. The principal user workflow is to supply Sentinel imagery to a model and receive encoded features for downstream EO tasks. Its documented design combines modality-specific patch embeddings, cross-sensor fusion, and a ViT encoder; pretraining also uses a student–teacher framework and dual-centering regularization. The single source file is only partially excerpted, so training internals and some model details are represented from README claims, not inspected implementation.
Main entry points: terrafm.py TerraFM models
Components 10
Inputs and Use
- Model user
- Sentinel imagery
Model Encoding
- TerraFM models
terrafm.py - Modality embeddings
terrafm.py - Cross-attention fusion
terrafm.py - ViT encoder
terrafm.py - Encoded features
terrafm.py
Pretraining Design
- Student-teacher training
- Dual-centering
EO Tasks
- EO downstream tasks
Connections 8
- Model user → TerraFM models: loads model
README.md - Sentinel imagery → TerraFM models: supplies input
README.md - TerraFM models → Modality embeddings: embeds modalities
README.md - Modality embeddings → Cross-attention fusion: feeds tokens
README.md - Cross-attention fusion → ViT encoder: fuses sensors
README.md - ViT encoder → Encoded features: encodes tokens
terrafm.py - Encoded features → EO downstream tasks: supports evaluation
README.md - Student-teacher training → Dual-centering: uses regularization
README.md