# google-deepmind/graphcast architecture

> Architecture diagram of the GitHub repository google-deepmind/graphcast, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-06-06). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/google-deepmind/graphcast
- Repository: https://github.com/google-deepmind/graphcast

## Overview

##### Project Type & Purpose
- Type: Open-source ML research package for medium-range global weather forecasting.
- Purpose: Provide implementations of two neural forecasting models (GraphCast and GenCast), data-prep utilities, training/inference wrappers, pretrained weights, and Colab demos.

##### Top-Level Components
a. Data Ingestion & Preprocessing
– Modules: data_utils.py, normalization.py, nan_cleaning.py, xarray_jax.py, xarray_tree.py
– Inputs: Raw ERA5/HRES Zarr datasets (ECMWF, Cloud Bucket)
b. Mesh & Graph Construction
– Modules: icosahedral_mesh.py, grid_mesh_connectivity.py, typed_graph.py, model_utils.py
– Output: TypedGraph objects (nodes + edges + features)
c. Core Model Definitions
– GraphCast path: graphcast.py, casting.py, solar_radiation.py
– GenCast path: gencast.py, denoiser.py, denoisers_base.py, dpm_solver_plus_plus_2s.py
– Common building blocks: deep_typed_graph_net.py, typed_graph_net.py, mlp.py, transformer.py, sparse_transformer.py
d. Training & Inference Wrappers
– Autoregressive training: autoregressive.py
– Inference rollout: rollout.py
– Predictor & sampler interfaces: predictor_base.py, samplers_base.py, samplers_utils.py
– Checkpointing: checkpoint.py
– Loss computation: losses.py
e. Demos & Notebooks
– gencast_demo.ipynb, gencast_demo_cloud_vm.ipynb, gencast_mini_demo.ipynb, graphcast_demo.ipynb
f. External Services & Dependencies
– Google Cloud Storage (pretrained weights, stats)
– ECMWF Zarr endpoints (ERA5, HRES)
– Python ecosystem: JAX, Haiku, Jraph, Dask, Xarray, etc.

##### Relationships & Data Flow
1. External data (ECMWF Zarr, Cloud Bucket) → Data Ingestion & Preprocessing
2. Preprocessed grid data → Mesh & Graph Construction → TypedGraph
3. TypedGraph → Core Model → One-step predictions
4. One-step outputs ↔ Autoregressive or Rollout wrappers → multi-step forecast
5. Forecasts → Loss module (during training) → gradients → checkpoint → repeat
6. Inference forecasts → Client/demos or persisted to Cloud Bucket

##### Architectural Patterns & Principles
- Modular layering: clear separation of data-prep, graph construction, model definition, and runtime wrappers.
- Interface-based design: predictor_base, denoisers_base, samplers_base allow swapping components (GraphCast vs GenCast, different samplers).
- Functional/nested data structures: JAX tree utilities (xarray_tree, checkpoint) for serialization and transformation.
- Graph-neural-network abstractions: TypedGraph → deep_typed_graph_net or sparse_transformer modules.

##### Minimum Elements
- Modules grouped by function and named explicitly.
- External integrations (datasets & storage).
- Notes on technologies/frameworks (JAX, Haiku, Jraph).

## Diagram (Mermaid)

```mermaid
flowchart TB
    %% External Systems
    ECMWFZ["ECMWF Zarr Endpoints"]:::ext
    GCS["Google Cloud Storage"]:::ext

    %% Data Layer
    subgraph "Data Layer"
        DU["data_utils.py"]:::data
        NORM["normalization.py"]:::data
        CLEAN["nan_cleaning.py"]:::data
        XJAX["xarray_jax.py"]:::data
        XTREE["xarray_tree.py"]:::data
    end

    %% Graph Layer
    subgraph "Graph Layer"
        IM["icosahedral_mesh.py"]:::data
        GMC["grid_mesh_connectivity.py"]:::data
        TG["typed_graph.py"]:::data
        MU["model_utils.py"]:::data
    end

    %% Model Layer
    subgraph "Model Layer"
        subgraph "GraphCast Core Model"
            GC["graphcast.py"]:::core
            CAST["casting.py"]:::core
            SOL["solar_radiation.py"]:::core
        end
        subgraph "GenCast Core Model"
            GC2["gencast.py"]:::core
            DEN["denoiser.py"]:::core
            DB["denoisers_base.py"]:::core
            DPS["dpm_solver_plus_plus_2s.py"]:::core
        end
        subgraph "Common Building Blocks"
            DTGN["deep_typed_graph_net.py"]:::data
            TGN["typed_graph_net.py"]:::data
            MLP["mlp.py"]:::data
            TRANS["transformer.py"]:::data
            SPARSE["sparse_transformer.py"]:::data
            SPUTIL["sparse_transformer_utils.py"]:::data
        end
    end

    %% Runtime Layer
    subgraph "Runtime Layer"
        AR["autoregressive.py"]:::runtime
        RO["rollout.py"]:::runtime
        PB["predictor_base.py"]:::runtime
        SB["samplers_base.py"]:::runtime
        SU["samplers_utils.py"]:::runtime
        CKPT["checkpoint.py"]:::runtime
        LOSS["losses.py"]:::runtime
    end

    %% Demos & Notebooks
    subgraph "Demos & Notebooks"
        D1["gencast_demo.ipynb"]:::demo
        D2["gencast_demo_cloud_vm.ipynb"]:::demo
        D3["gencast_mini_demo.ipynb"]:::demo
        D4["graphcast_demo.ipynb"]:::demo
    end

    %% Data Flow
    ECMWFZ --> DU
    GCS --> DU
    DU --> IM
    DU --> NORM
    DU --> CLEAN
    DU --> XJAX
    DU --> XTREE
    IM --> TG
    GMC --> TG
    TG --> GC
    TG --> GC2
    DTGN --> GC
    TGN --> GC
    MLP --> GC
    TRANS --> GC
    SPARSE --> GC
    SPUTIL --> GC
    DTGN --> GC2
    TGN --> GC2
    MLP --> GC2
    TRANS --> GC2
    SPARSE --> GC2
    SPUTIL --> GC2
    GC --> AR
    CAST --> AR
    SOL --> AR
    GC2 --> AR
    DEN --> AR
    DB --> AR
    DPS --> AR
    AR --> LOSS
    LOSS --> CKPT
    CKPT --> AR
    AR --> RO
    RO --> GCS
    RO --> D1
    RO --> D2
    RO --> D3
    RO --> D4

    %% Click Events
    click DU "https://github.com/google-deepmind/graphcast/blob/main/graphcast/data_utils.py"
    click NORM "https://github.com/google-deepmind/graphcast/blob/main/graphcast/normalization.py"
    click CLEAN "https://github.com/google-deepmind/graphcast/blob/main/graphcast/nan_cleaning.py"
    click XJAX "https://github.com/google-deepmind/graphcast/blob/main/graphcast/xarray_jax.py"
    click XTREE "https://github.com/google-deepmind/graphcast/blob/main/graphcast/xarray_tree.py"
    click IM "https://github.com/google-deepmind/graphcast/blob/main/graphcast/icosahedral_mesh.py"
    click GMC "https://github.com/google-deepmind/graphcast/blob/main/graphcast/grid_mesh_connectivity.py"
    click TG "https://github.com/google-deepmind/graphcast/blob/main/graphcast/typed_graph.py"
    click MU "https://github.com/google-deepmind/graphcast/blob/main/graphcast/model_utils.py"
    click GC "https://github.com/google-deepmind/graphcast/blob/main/graphcast/graphcast.py"
    click CAST "https://github.com/google-deepmind/graphcast/blob/main/graphcast/casting.py"
    click SOL "https://github.com/google-deepmind/graphcast/blob/main/graphcast/solar_radiation.py"
    click GC2 "https://github.com/google-deepmind/graphcast/blob/main/graphcast/gencast.py"
    click DEN "https://github.com/google-deepmind/graphcast/blob/main/graphcast/denoiser.py"
    click DB "https://github.com/google-deepmind/graphcast/blob/main/graphcast/denoisers_base.py"
    click DPS "https://github.com/google-deepmind/graphcast/blob/main/graphcast/dpm_solver_plus_plus_2s.py"
    click DTGN "https://github.com/google-deepmind/graphcast/blob/main/graphcast/deep_typed_graph_net.py"
    click TGN "https://github.com/google-deepmind/graphcast/blob/main/graphcast/typed_graph_net.py"
    click MLP "https://github.com/google-deepmind/graphcast/blob/main/graphcast/mlp.py"
    click TRANS "https://github.com/google-deepmind/graphcast/blob/main/graphcast/transformer.py"
    click SPARSE "https://github.com/google-deepmind/graphcast/blob/main/graphcast/sparse_transformer.py"
    click SPUTIL "https://github.com/google-deepmind/graphcast/blob/main/graphcast/sparse_transformer_utils.py"
    click AR "https://github.com/google-deepmind/graphcast/blob/main/graphcast/autoregressive.py"
    click RO "https://github.com/google-deepmind/graphcast/blob/main/graphcast/rollout.py"
    click PB "https://github.com/google-deepmind/graphcast/blob/main/graphcast/predictor_base.py"
    click SB "https://github.com/google-deepmind/graphcast/blob/main/graphcast/samplers_base.py"
    click SU "https://github.com/google-deepmind/graphcast/blob/main/graphcast/samplers_utils.py"
    click CKPT "https://github.com/google-deepmind/graphcast/blob/main/graphcast/checkpoint.py"
    click LOSS "https://github.com/google-deepmind/graphcast/blob/main/graphcast/losses.py"

    %% Styles
    classDef data fill:#E0F7FA,stroke:#00838F,color:#004D40
    classDef core fill:#C8E6C9,stroke:#2E7D32,color:#1B5E20
    classDef runtime fill:#FFE0B2,stroke:#EF6C00,color:#E65100
    classDef ext fill:#ECEFF1,stroke:#90A4AE,color:#607D8B
    classDef demo fill:#F3E5F5,stroke:#8E24AA,color:#4A148C
```
