How lmissher/stgnn works
Project type and purpose
• This is a machine-learning research codebase (PyTorch implementation) for traffic-flow forecasting using a spatial-temporal graph neural network. It is not a full-stack web application but rather an ML pipeline with data processing, model definition, training, and evaluation.
File structure
• Top-level scripts:
• model.py – defines the STGNN architecture (layers, graph attention, wavelet transforms).
• train.py – orchestrates dataset loading, training loop, checkpointing, and evaluation.
• utils.py – helper routines for data preprocessing, batching, metrics, possibly graph construction.
• Configuration: arguments are passed into train.py (e.g., hyperparameters, file paths).
• No dedicated “data/” or “inference/” directories—those are handled inline in train.py via utils.
README for additional insights
• The README points to two research papers describing the algorithmic advances and expected workflow:
(1) spectral graph attention for disentangled ST forecasting
(2) large-scale extension with transformers.
a. Main components
• Data Sources (Raw traffic sensor readings, CSV/Parquet files)
• Data Ingestion & Preprocessing (utils.py)
• Graph Construction Engine (spatial topology builder)
• Model Definition (model.py – STGNN layers)
• Training & Validation Loop (train.py)
• Experiment Tracking & Checkpoint Store (filesystem or S3/DB)
• Evaluation & Metrics (organized in train.py / utils.py)
• (Optional) Inference / Serving Module for online predictions
b. Relationships & interactions
1. Data Sources → Data Ingestion & Preprocessing
2. Preprocessed data → Graph Construction → Batches of tensors
3. Batches → Model Definition → Forward/Backward pass
4. Training Loop → Checkpoint Store (periodic saves)
5. Validation data → Metrics → Experiment Tracking
6. Final model artifacts → Optional Serving Layer
c. Architectural patterns & design principles
• Pipeline pattern: sequential data flow through modules
• Modular separation: the “utils” layer decouples data logic from training logic
• Compute abstraction: model definition is isolated from the orchestration loop
• Checkpointing for reproducibility
d. Key technologies & libraries
• PyTorch – core deep-learning framework
• Python standard libs (argparse, logging)
• Scientific stack (NumPy, pandas)
• (Optional) MLflow/TensorBoard for experiment tracking
Detail-level advice
• Break the pipeline into as many discrete stages as make sense (ingest, clean, graph-init, model-forward, backprop, eval).
• Optional external services (GPU cluster, S3, MLflow server) if used in your environment.