# cmeo97/nowcastinggpt architecture

> Architecture diagram of the GitHub repository cmeo97/nowcastinggpt, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-05-12). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/cmeo97/nowcastinggpt
- Repository: https://github.com/cmeo97/nowcastinggpt

## Overview

##### Project type and purpose
- This is a research-oriented, full-stack machine-learning system for extreme precipitation nowcasting.
- It consists of data ingestion & preprocessing, model training (VQ-VAE tokeniser + autoregressive Transformer), evaluation/prediction, configuration management, and experiment logging.

##### File structure
- Top-level directories:
- Dataset_CSV_Files (raw CSV pointers to radar images)
- config (YAML configs for tokenizer and world_model/trainer)
- asset (static images for documentation)
- src (all Python source code, subdivided into preprocessing, model definitions, training, prediction, and utilities)
- Key scripts:
- src/Resize_Images.py → converts raw images → NumPy arrays
- src/collector.py → builds PyTorch DataLoaders
- src/main.py & trainer.sh → orchestrate end-to-end training and evaluation
- src/make_prediction.py & make_reconstructions.py → generate and save predicted/reconstructed frames
- src/trainer.py → training loop wrapper (integrates with Weights & Biases)
- Models in src/models:
- tokenizer (VQ-VAE nets, LPIPS loss, discrete tokeniser)
- transformer (autoregressive world model + classifier head)
- Config files: define hyperparameters, paths, wandb settings.

##### README for additional insights
- The system is built around:
- A VQ-VAE “tokeniser” that compresses radar frames into discrete codes
- A Transformer “world model” that autoregressively predicts future token sequences
- An Extreme Value Loss (EVL) regularization module
- Training and evaluation are driven by YAML configurations and a Bash wrapper.
- Logging to Weights & Biases is built-in.

##### Components & Interactions
a. Main components:
1. Data Source: CSV files pointing to radar image archives
2. Preprocessing: Resize_Images.py → NumPy arrays
3. Data Loader: Collector class → PyTorch DataLoaders
4. Model Stack:
- VQ-VAE Tokeniser (src/models/tokenizer/*)
- Autoregressive Transformer (src/models/transformer.py + classifier)
- Extreme Value Loss module (integrated into trainer)
5. Training Orchestrator:
- main.py & trainer.py
- trainer.sh for shell control
6. Logging & Monitoring: Weights & Biases
7. Prediction/Evaluation:
- make_prediction.py → future frames
- make_reconstructions.py → VQ-VAE reconstructions
8. Outputs:
- outputs/<date>/media/predictions
- outputs/<date>/media/reconstructions
9. Configuration Store: config/*.yaml
10. Utilities: src/utils.py, kv_caching, slicer, etc.

b. Relationships & data flow:
- Raw CSV metadata → (Resize_Images) → NumPy arrays
- NumPy arrays → (Collector) → DataLoader batches
- Batches → (VQ-VAE) → discrete tokens → (Transformer) → predicted token sequences
- Tokens → (Reconstruction) → frames → saved to outputs
- Trainer orchestrates forward/backward passes, EVL computation, calls wandb API.

c. Architectural patterns:
- Modular pipeline (ETL → modeling → evaluation)
- Clear separation of concerns: preprocessing, data loading, model definition, training logic, config
- Config-driven execution (YAML) for reproducibility
- Use of a script wrapper (trainer.sh) for remote/nohup execution

d. Key technologies & libraries:
- Python 3.8.16, PyTorch 1.11.0, Torchvision, Torchaudio
- PyTorch Lightning utilities, torchmetrics, pycm
- Weights & Biases for logging
- YAML for configuration

## Diagram (Mermaid)

```mermaid
flowchart TB
    %% Config Layer
    subgraph "Config Layer"
        C1["tokenizer/default.yaml"]:::config
        C2["world_model/default.yaml"]:::config
        C3["trainer.yaml"]:::config
    end

    %% Data Layer
    subgraph "Data Layer"
        DS["CSV Metadata"]:::data
    end

    %% Preprocessing Layer
    subgraph "Preprocessing Layer"
        RI["Resize_Images.py"]:::compute
        NP["NumPy Arrays"]:::data
        DL["collector.py"]:::compute
    end

    %% Model Layer
    subgraph "Model Layer"
        subgraph "VQ-VAE Tokeniser"
            NTS["nets.py"]:::compute
            LP["lpips.py"]:::compute
            TOK["tokenizer.py"]:::compute
        end
        subgraph "Transformer World Model"
            TRF["transformer.py"]:::compute
            WM["world_model.py"]:::compute
            CLF["transformer_classifier.py"]:::compute
        end
        subgraph "Utilities"
            UT["utils.py"]:::compute
            KV["kv_caching.py"]:::compute
            SL["slicer.py"]:::compute
        end
    end

    %% Training & Logging Layer
    subgraph "Training & Logging Layer"
        MP["main.py"]:::compute
        TRN["trainer.py"]:::compute
        SH["trainer.sh"]:::compute
        WB["Weights & Biases"]:::logging
    end

    %% Evaluation/Serving Layer
    subgraph "Evaluation/Serving Layer"
        PR["make_prediction.py"]:::compute
        RC["make_reconstructions.py"]:::compute
        OUT["Outputs/<date>/media"]:::data
    end

    %% Data Flow
    DS -->|"raw CSV"| RI
    RI -->|"creates"| NP
    NP -->|"batches"| DL
    DL -->|"feeds"| NTS
    NTS -->|"encodes"| TOK
    TOK -->|"tokens"| TRF
    TRF -->|"predictions"| CLF
    CLF -->|"features"| WM
    DL -->|"feeds"| MP
    NTS -->|"models"| MP
    TOK -->|"models"| MP
    TRF -->|"models"| MP
    WM -->|"models"| MP
    CLF -->|"models"| MP
    MP -->|"orchestrates"| TRN
    TRN -->|"logs metrics"| WB
    MP -->|"called by"| SH
    PR -->|"uses models"| NTS
    PR -->|"uses models"| TRF
    PR -->|"outputs frames"| OUT
    RC -->|"reconstructs"| OUT

    %% Config Connections
    C1 --> MP
    C2 --> MP
    C3 --> MP
    C1 --> PR
    C2 --> PR
    C3 --> PR
    C1 --> RC
    C2 --> RC
    C3 --> RC

    %% Click Events
    click DS "https://github.com/cmeo97/nowcastinggpt/tree/main/Dataset_CSV_Files/"
    click RI "https://github.com/cmeo97/nowcastinggpt/blob/main/src/Resize_Images.py"
    click DL "https://github.com/cmeo97/nowcastinggpt/blob/main/src/collector.py"
    click NTS "https://github.com/cmeo97/nowcastinggpt/blob/main/src/models/tokenizer/nets.py"
    click LP "https://github.com/cmeo97/nowcastinggpt/blob/main/src/models/tokenizer/lpips.py"
    click TOK "https://github.com/cmeo97/nowcastinggpt/blob/main/src/models/tokenizer/tokenizer.py"
    click TRF "https://github.com/cmeo97/nowcastinggpt/blob/main/src/models/transformer.py"
    click WM "https://github.com/cmeo97/nowcastinggpt/blob/main/src/models/world_model.py"
    click CLF "https://github.com/cmeo97/nowcastinggpt/blob/main/src/models/transformer_classifier.py"
    click TRN "https://github.com/cmeo97/nowcastinggpt/blob/main/src/trainer.py"
    click MP "https://github.com/cmeo97/nowcastinggpt/blob/main/src/main.py"
    click SH "https://github.com/cmeo97/nowcastinggpt/blob/main/trainer.sh"
    click PR "https://github.com/cmeo97/nowcastinggpt/blob/main/src/make_prediction.py"
    click RC "https://github.com/cmeo97/nowcastinggpt/blob/main/src/make_reconstructions.py"
    click C1 "https://github.com/cmeo97/nowcastinggpt/blob/main/config/tokenizer/default.yaml"
    click C2 "https://github.com/cmeo97/nowcastinggpt/blob/main/config/world_model/default.yaml"
    click C3 "https://github.com/cmeo97/nowcastinggpt/blob/main/config/trainer.yaml"

    %% Styles
    classDef data fill:#ADD8E6,stroke:#000;
    classDef compute fill:#90EE90,stroke:#000;
    classDef config fill:#FFA500,stroke:#000;
    classDef logging fill:#DDA0DD,stroke:#000;
```
