# mbzuai-oryx/terrafm architecture

> Architecture diagram of the GitHub repository mbzuai-oryx/terrafm, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-10-03). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/mbzuai-oryx/terrafm
- Repository: https://github.com/mbzuai-oryx/terrafm

## Overview

TerraFM is a pretrained multisensor Earth-observation vision model, exposed as standalone Base and Large PyTorch models. The principal user workflow is to supply Sentinel imagery to a model and receive encoded features for downstream EO tasks. Its documented design combines modality-specific patch embeddings, cross-sensor fusion, and a ViT encoder; pretraining also uses a student–teacher framework and dual-centering regularization. The single source file is only partially excerpted, so training internals and some model details are represented from README claims, not inspected implementation.

Main entry points: [`terrafm.py`](https://github.com/mbzuai-oryx/terrafm/blob/master/terrafm.py) (TerraFM models)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_input["Inputs and Use"]
  node_user(("Model user"))
  node_imagery(("Sentinel imagery"))
end

subgraph group_model["Model Encoding"]
  node_model_api["TerraFM models<br/>[terrafm.py]"]
  node_patch["Modality embeddings<br/>[terrafm.py]"]
  node_fusion["Cross-attention fusion<br/>[terrafm.py]"]
  node_vit["ViT encoder<br/>[terrafm.py]"]
  node_features["Encoded features<br/>[terrafm.py]"]
end

subgraph group_training["Pretraining Design"]
  node_pretraining["Student-teacher training"]
  node_centering["Dual-centering"]
end

subgraph group_tasks["EO Tasks"]
  node_eo_tasks["EO downstream tasks"]
end

node_user -->|"loads model"| node_model_api
node_imagery -->|"supplies input"| node_model_api
node_model_api -->|"embeds modalities"| node_patch
node_patch -->|"feeds tokens"| node_fusion
node_fusion -->|"fuses sensors"| node_vit
node_vit -->|"encodes tokens"| node_features
node_features -->|"supports evaluation"| node_eo_tasks
node_pretraining -->|"uses regularization"| node_centering

click node_model_api "https://github.com/mbzuai-oryx/terrafm/blob/master/terrafm.py"
click node_patch "https://github.com/mbzuai-oryx/terrafm/blob/master/terrafm.py"
click node_fusion "https://github.com/mbzuai-oryx/terrafm/blob/master/terrafm.py"
click node_vit "https://github.com/mbzuai-oryx/terrafm/blob/master/terrafm.py"
click node_features "https://github.com/mbzuai-oryx/terrafm/blob/master/terrafm.py"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_user,node_imagery toneBlue
class node_model_api,node_patch,node_fusion,node_vit,node_features toneAmber
class node_pretraining,node_centering toneMint
class node_eo_tasks toneRose
```

## Components

### Inputs and Use

- **Model user**
- **Sentinel imagery**

### Model Encoding

- [TerraFM models](https://github.com/mbzuai-oryx/terrafm/blob/master/terrafm.py): `terrafm.py`
- [Modality embeddings](https://github.com/mbzuai-oryx/terrafm/blob/master/terrafm.py): `terrafm.py`
- [Cross-attention fusion](https://github.com/mbzuai-oryx/terrafm/blob/master/terrafm.py): `terrafm.py`
- [ViT encoder](https://github.com/mbzuai-oryx/terrafm/blob/master/terrafm.py): `terrafm.py`
- [Encoded features](https://github.com/mbzuai-oryx/terrafm/blob/master/terrafm.py): `terrafm.py`

### Pretraining Design

- **Student-teacher training**
- **Dual-centering**

### EO Tasks

- **EO downstream tasks**

## Connections

- Model user → TerraFM models: loads model (evidence: `README.md`)
- Sentinel imagery → TerraFM models: supplies input (evidence: `README.md`)
- TerraFM models → Modality embeddings: embeds modalities (evidence: `README.md`)
- Modality embeddings → Cross-attention fusion: feeds tokens (evidence: `README.md`)
- Cross-attention fusion → ViT encoder: fuses sensors (evidence: `README.md`)
- ViT encoder → Encoded features: encodes tokens (evidence: `terrafm.py`)
- Encoded features → EO downstream tasks: supports evaluation (evidence: `README.md`)
- Student-teacher training → Dual-centering: uses regularization (evidence: `README.md`)
