How bbbbihr/nowcasting-of-extreme-precipitation works
Project Type and Purpose
• This is a scientific machine‐learning pipeline for extreme precipitation nowcasting and detection. It bundles:
• A PySTEPS‐based baseline evaluation (Benchmark_test)
• An event‐selection and data‐analysis module
• A two‐stage deep‐learning model (VQGAN + Transformer)
• A shared RainyMotion library for data slicing, metrics, and utilities
• Raw radar/precipitation files (e.g. NetCDF/HDF5)
• Catchment definitions (mask .npy)
• Catchment_slice_RM.py (data slicing)
• metrics.py (evaluation functions)
• models.py (baseline or auxiliary models)
• utils.py (I/O helpers)
• event_selection_extreme_opti.py → computes accumulation & thresholds
• event_selection_function.py → splits into train/val/test sets
• event_analysis.py → basic statistics & visualization
• HPCrunScripts/*.py → scripts for running deterministic & probabilistic nowcasts
• config/ → PySTEPS configuration
• vqgan+transformer/vqgan_training.ipynb → trains the VQGAN encoder/decoder
• notebook or script that takes radar frames + pretrained VQGAN → latent token sequences
• transformer_training.ipynb → trains autoregressive transformer on token sequences
• experiment_nowcasting.ipynb → generates nowcasting outputs
• experiment_extreme_detection.ipynb → detects extreme events
• Results saved as .npy or visualizations
• README.md, docs/ folders, environment.yml, setup.py
1. Data Sources → RainyMotion Library (slicing, masking)
2. → Event Selection & Data Analysis → generates labeled train/val/test datasets
3. → VQGAN Model Training → outputs VQGAN model artifact
4. VQGAN model + Data → Tokenization → token sequences
5. → Transformer Model Training → outputs Transformer model artifact
6. → Experiment & Evaluation → nowcast maps and extreme‐event detections
7. Separately, Data Sources + PySTEPS Benchmark Module → baseline nowcasting results
Architectural Patterns & Principles
• Modular pipeline: each stage encapsulated in its own folder/notebook
• Reusable library (RainyMotion) for shared code
• Two‐stage deep‐learning (representation learning + sequence modeling)
• Separation of benchmark (PySTEPS) and research model