# maxaragon/skygpt architecture

> Architecture diagram of the GitHub repository maxaragon/skygpt, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-04-18). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/maxaragon/skygpt
- Repository: https://github.com/maxaragon/skygpt

## Overview

##### Project type and high‑level purpose
- This is a research‑oriented machine‑learning pipeline for ultra‑short‑term solar forecasting.
- It has two major stages:
– Stochastic sky‑video prediction (generating future sky frames)
– PV output forecasting (mapping images to power predictions)

##### File structure to extract components
- /codes/video_prediction/
– ConvLSTM (deterministic baseline)
– PhyDNetGAN (physics+GAN baseline)
– SkyGPT (VQ‑VAE + physics‑informed transformer)
– eval/ (quantitative evaluation notebooks)
- /codes/pv_output_prediction/
– SUNSET (AlexNet‑style CNN)
– UNet (modified for power regression)
- Data artifacts and dependencies
– video_prediction_dataset.hdf5, test_set_2019nov_dec.hdf5
– times_curr_*.npy
– requirements.txt (Python 3.6, PyTorch 1.8.1, TensorFlow 2.4.1)
- Figures and README describe framework and data flow

##### Sketch the main components
A. Data Storage
- Raw dataset (HDF5 files on local disk or Google Drive)
- Metadata (timestamps .npy)
B. Preprocessing
- reformat_input.py (SkyGPT input)
- any notebook cells that load, normalize, and batch images/PV
C. Video‐Prediction Models
- ConvLSTM module
- PhyDNet + GAN module
- SkyGPT module:
– VQ‑VAE (encoder/decoder)
– PhyCell (physics constraint)
– Transformer (auto‑regressive token model)
- Training scripts: training_vqvae.py, training_transformer.py
- Sample generation script: sample_gen.py
D. PV Output Prediction Models
- SUNSET training notebook (AlexNet CNN)
- U‑Net training notebook (modified UNet)
E. Evaluation & Comparison
- quant_eval_*.ipynb (compute MAE, MSE, VGG similarity)
- comp_all_models.ipynb (aggregate comparisons)
F. Hardware & Environment
- GPU cluster (NVIDIA A100)
- Python environments (TF, PyTorch)

##### Architectural patterns and technologies
- Modular ML pipeline (each model lives in its own subfolder + scripts)
- Hybrid deep‐learning frameworks (PyTorch for video, TensorFlow for PV)
- Discrete latent modelling (VQ‑VAE) + physics‐informed transformer
- Notebook‑driven orchestration for experiments and result visualization

## Diagram (Mermaid)

```mermaid
flowchart TD
    %% Data Storage
    DataStore["Raw Datasets (HDF5) & Metadata (.npy)"]:::data

    %% Preprocessing
    Preprocessing["Reformat Input (reformat_input.ipynb)"]:::notebook
    click Preprocessing "https://github.com/maxaragon/skygpt/blob/main/codes/video_prediction/SkyGPT/script/reformat_input.ipynb"

    %% Video Prediction Subsystem
    subgraph "Video Prediction"
        direction TB
        subgraph "Baseline Models"
            direction TB
            ConvLSTM["ConvLSTM (models.py)"]:::module
            click ConvLSTM "https://github.com/maxaragon/skygpt/blob/main/codes/video_prediction/ConvLSTM/models.py"
            ConvLSTM_NB["ConvLSTM.ipynb"]:::notebook
            click ConvLSTM_NB "https://github.com/maxaragon/skygpt/blob/main/codes/video_prediction/ConvLSTM/ConvLSTM.ipynb"
            PhyDNetGAN["PhyDNetGAN"]:::module
            PhyDNetGAN_M1["models_v2.py"]:::module
            click PhyDNetGAN_M1 "https://github.com/maxaragon/skygpt/blob/main/codes/video_prediction/PhyDNetGAN/models_v2.py"
            PhyDNetGAN_M2["constrain_moments.py"]:::module
            click PhyDNetGAN_M2 "https://github.com/maxaragon/skygpt/blob/main/codes/video_prediction/PhyDNetGAN/constrain_moments.py"
            PhyDNetGAN_NB["PhyDNetGAN.ipynb"]:::notebook
            click PhyDNetGAN_NB "https://github.com/maxaragon/skygpt/blob/main/codes/video_prediction/PhyDNetGAN/PhyDNetGAN.ipynb"
        end

        subgraph "SkyGPT Model (PyTorch)"
            direction TB
            VQVAE["VQ-VAE (vqvae.py)"]:::module
            click VQVAE "https://github.com/maxaragon/skygpt/blob/main/codes/video_prediction/SkyGPT/skygpt/vqvae.py"
            PhyCell["PhyCell (constrain_moments.py)"]:::module
            click PhyCell "https://github.com/maxaragon/skygpt/blob/main/codes/video_prediction/SkyGPT/skygpt/constrain_moments.py"
            Attention["Attention (attention.py)"]:::module
            click Attention "https://github.com/maxaragon/skygpt/blob/main/codes/video_prediction/SkyGPT/skygpt/attention.py"
            Transformer["Transformer (gpt.py)"]:::module
            click Transformer "https://github.com/maxaragon/skygpt/blob/main/codes/video_prediction/SkyGPT/skygpt/gpt.py"
            ResNet["ResNet (resnet.py)"]:::module
            click ResNet "https://github.com/maxaragon/skygpt/blob/main/codes/video_prediction/SkyGPT/skygpt/resnet.py"
            DataUtils["Data Utils (data.py)"]:::module
            click DataUtils "https://github.com/maxaragon/skygpt/blob/main/codes/video_prediction/SkyGPT/skygpt/data.py"
            Utils["Utils (utils.py)"]:::module
            click Utils "https://github.com/maxaragon/skygpt/blob/main/codes/video_prediction/SkyGPT/skygpt/utils.py"

            subgraph "Training & Inference Scripts"
                direction TB
                TrainVQVAE["train_vqvae.py"]:::notebook
                click TrainVQVAE "https://github.com/maxaragon/skygpt/blob/main/codes/video_prediction/SkyGPT/script/train_vqvae.py"
                TrainTransformer["train_transformer.py"]:::notebook
                click TrainTransformer "https://github.com/maxaragon/skygpt/blob/main/codes/video_prediction/SkyGPT/script/train_transformer.py"
                SampleGen["sample_gen.ipynb"]:::notebook
                click SampleGen "https://github.com/maxaragon/skygpt/blob/main/codes/video_prediction/SkyGPT/script/sample_gen.ipynb"
            end
        end
    end

    %% Generated Videos
    GeneratedVideos["Generated Sky Videos"]:::module

    %% PV Prediction Subsystem
    subgraph "PV Output Prediction"
        direction TB
        SUNSET["SUNSET_PV_forecast.ipynb"]:::notebook
        click SUNSET "https://github.com/maxaragon/skygpt/blob/main/codes/pv_output_prediction/SUNSET_PV_forecast.ipynb"
        UNet["UNet_sky_image_PV_mapping.ipynb"]:::notebook
        click UNet "https://github.com/maxaragon/skygpt/blob/main/codes/pv_output_prediction/UNet_sky_image_PV_mapping.ipynb"
    end

    %% Evaluation & Comparison
    subgraph "Evaluation & Comparison"
        direction TB
        CompareAll["comp_all_models.ipynb"]:::notebook
        click CompareAll "https://github.com/maxaragon/skygpt/blob/main/codes/video_prediction/eval/comp_all_models.ipynb"
        QuanSkyGPT["quan_eval_SkyGPT.ipynb"]:::notebook
        click QuanSkyGPT "https://github.com/maxaragon/skygpt/blob/main/codes/video_prediction/eval/quan_eval_SkyGPT.ipynb"
        QuanVideoGPT["quan_eval_VideoGPT.ipynb"]:::notebook
        click QuanVideoGPT "https://github.com/maxaragon/skygpt/blob/main/codes/video_prediction/eval/quan_eval_VideoGPT.ipynb"
        QuanExcl["quan_eval_excld_VideoGPT_and_SkyGPT.ipynb"]:::notebook
        click QuanExcl "https://github.com/maxaragon/skygpt/blob/main/codes/video_prediction/eval/quan_eval_excld_VideoGPT_and_SkyGPT.ipynb"
    end

    %% Connections
    DataStore --> Preprocessing
    Preprocessing --> ConvLSTM
    Preprocessing --> PhyDNetGAN
    Preprocessing --> VQVAE
    ConvLSTM --> GeneratedVideos
    PhyDNetGAN --> GeneratedVideos
    SampleGen --> GeneratedVideos
    GeneratedVideos --> SUNSET
    GeneratedVideos --> UNet
    SUNSET --> CompareAll
    UNet --> CompareAll
    CompareAll --> QuanSkyGPT
    CompareAll --> QuanVideoGPT
    CompareAll --> QuanExcl

    %% Styles
    classDef data fill:#bbdefb,stroke:#0d47a1,color:#0d47a1
    classDef module fill:#c8e6c9,stroke:#1b5e20,color:#1b5e20
    classDef notebook fill:#ffe0b2,stroke:#e65100,color:#e65100
```
