# vitusbenson/greenearthnet architecture

> Architecture diagram of the GitHub repository vitusbenson/greenearthnet, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-06-18). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/vitusbenson/greenearthnet
- Repository: https://github.com/vitusbenson/greenearthnet

## Overview

##### Project Type and Purpose
- This is a research-oriented machine-learning codebase for multi-modal geospatial vegetation forecasting.
- It provides scripts to train, test (inference), and evaluate a variety of spatio-temporal models (e.g., Contextformer, ConvLSTM, PredRNN, SimVP) on the GreenEarthNet dataset, plus a small suite of pixel-wise baseline models.
- It is not a full-stack web app but a pipeline-style system: data ingestion → preprocessing (cloud masking) → model training/inference → evaluation.

##### High-Level File/Directory Analysis
- Top‐level scripts:
- train.py – launches training according to a YAML config
- test.py – runs inference, producing predicted output cubes
- eval.py – computes performance metrics on saved predictions
- cloudmask/train_cloudmask_l2argbnir.py – trains a cloud-masking U-Net
- model_configs/… – nested directories of YAML configuration files for each model variant (Contextformer, ConvLSTM, etc.)
- model_pixelwise/… – Python scripts implementing simple baselines (persistence, climatology, Kalman+XGB, etc.)
- Dependencies: earthnet & earthnet-minicuber for data handling; earthnet-models-pytorch and PyTorch Lightning for model implementations and training loops.

##### Main Components
A. Data Layer
- Raw Data Store (“GreenEarthNet Dataset”) – downloaded via earthnet toolkit, stored on disk (data_dir)
- Data Generation Module (“EarthNet Minicuber”) – produces aligned “minicubes” of multi-modal geospatial inputs
B. Preprocessing Layer
- Cloud Mask Algorithm – either uses pretrained model from earthnet-minicuber or trains new one via cloudmask/train_cloudmask_l2argbnir.py
C. Configuration Layer
- model_configs directory – YAML files specifying model architecture, hyperparameters, seed, weather tokens, etc.
D. Modeling Layer
- External Model Library – earthnet-models-pytorch contains implementations of Contextformer, ConvLSTM, PredRNN, SimVP, etc.
- Pixelwise Baselines – small scripts under model_pixelwise/ for non-deep-learning baselines
E. Training Pipeline
- train.py – reads a config, instantiates a PyTorch Lightning Trainer and DataModule, kicks off training, checkpoints weights
- Logging – TensorBoard, CLI outputs
F. Inference Pipeline
- test.py – loads checkpoint + YAML config, runs forward pass on test splits, writes predictions to disk
G. Evaluation Pipeline
- eval.py – takes ground-truth dataset and predictions, computes accuracy metrics, supports optional comparison directory

##### Relationships and Data Flows
- Config files feed into both Training and Inference pipelines
- Inference pipeline reads from Checkpoints + Raw Data → Prediction outputs
- Evaluation reads from Raw Data + Predictions → Scores
- Pixelwise Baselines bypass Deep Learning models: Raw Data → model_pixelwise scripts → Predictions → Eval

##### Architectural Patterns and Principles
- Pipeline architecture: clearly separated stages (ingestion, preprocessing, training, inference, evaluation)
- Config-driven design: no hard-coded hyperparameters; all behavior controlled by YAML.
- Modularization: distinct modules for deep models vs. pixel-wise baselines vs. cloud masking.
- Use of external libraries for domain-specific logic (earthnet, earthnet-minicuber, earthnet-models-pytorch).

- Clear, consistent labels on every component (e.g., “train.py (PyTorch Lightning Trainer)”)

## Diagram (Mermaid)

```mermaid
flowchart TB
    %% Data Layer
    subgraph "Data Layer"
        direction TB
        DS["GreenEarthNet Dataset"]:::dataStore
        EM["EarthNet Minicuber"]:::external
    end

    %% Preprocessing Layer
    subgraph "Preprocessing Layer"
        direction TB
        CM["Cloud Mask Algorithm"]:::process
    end

    %% Configuration Layer
    subgraph "Configuration Layer"
        direction TB
        MC["model_configs/ (YAML Configs)"]:::config
    end

    %% Modeling Layer
    subgraph "Modeling Layer"
        direction TB
        EL["earthnet-models-pytorch"]:::external
        PB["Pixelwise Baseline Models"]:::process
    end

    %% Training Stage
    subgraph "Training Stage"
        direction TB
        T["train.py (PyTorch Lightning Trainer)"]:::cli
        CKP["Model Checkpoints (.ckpt)"]:::dataStore
        T -->|"checkpoints"| CKP
    end

    %% Inference Stage
    subgraph "Inference Stage"
        direction TB
        I["test.py (Inference)"]:::cli
        PRED["Predictions (.npy)"]:::dataStore
        I -->|"writes .npy"| PRED
    end

    %% Evaluation Stage
    subgraph "Evaluation Stage"
        direction TB
        E["eval.py (Metrics Computation)"]:::cli
        SCORE["Evaluation Scores"]:::dataStore
        E -->|"outputs metrics"| SCORE
    end

    %% Data & Control Flows
    DS -->|"reads raw .zarr"| EM
    EM -->|"produces minicubes"| CM
    CM -->|"cloud-masked cubes"| T
    MC -->|"reads .yaml"| T
    MC -->|"reads .yaml"| I
    CKP -->|"loads .ckpt"| I
    DS -->|"reads raw .zarr"| I
    DS -->|"reads raw .zarr"| E
    PRED -->|"reads .npy"| E

    %% Pixelwise branch
    DS -->|"reads raw .zarr"| PB
    PB -->|"writes .npy"| PRED

    %% External model lib usage
    T -->|"uses models"| EL

    %% Click events
    click CM "https://github.com/vitusbenson/greenearthnet/blob/main/cloudmask/train_cloudmask_l2argbnir.py"
    click MC "https://github.com/vitusbenson/greenearthnet/tree/main/model_configs/"
    click PB "https://github.com/vitusbenson/greenearthnet/tree/main/model_pixelwise/"
    click T "https://github.com/vitusbenson/greenearthnet/blob/main/train.py"
    click I "https://github.com/vitusbenson/greenearthnet/blob/main/test.py"
    click E "https://github.com/vitusbenson/greenearthnet/blob/main/eval.py"

    %% Styles
    classDef dataStore fill:#D0E4F5,stroke:#036,stroke-width:1px
    classDef process fill:#D9F5D3,stroke:#093,stroke-width:1px
    classDef cli fill:#F5E6D0,stroke:#E78A00,stroke-width:1px
    classDef external fill:#FFF,stroke:#999,stroke-dasharray: 5 5,stroke-width:1px
    classDef config fill:#F0D9F5,stroke:#703090,stroke-width:1px
```
