How guoshnbjtu/astgnn works
Project Type and Purpose
• This is a machine-learning pipeline project implementing ASTGNN (Adaptive Spatio-Temporal Graph Neural Network) in PyTorch, aimed at traffic forecasting on PEMS datasets.
• It takes raw traffic sensor data (CSV/NPZ), preprocesses it into a format suitable for graph-based neural networks, trains the ASTGNN model, and reports evaluation metrics.
File Structure Analysis
• Top-Level Scripts:
• prepareData.py – data preprocessing and graph construction
• train_ASTGNN.py – model training and evaluation
• Directories:
• configurations/ – experiment configurations per dataset (PEMS03, PEMS04, etc.)
• data/ – raw and cached datasets (CSV, NPZ, TXT)
• lib/ – utility modules (metrics.py for evaluation metrics, utils.py for helper functions)
• model/ – ASTGNN model definition (ASTGNN.py)
• One-shot pipeline: config → prepareData → model training using the same config.
README Insights
• Describes two experiment modes (ASTGNN and ASTGNN(p)) toggled by different config files.
• Shows usage commands for preprocessing and training with CUDA.
• Emphasizes configurable hyperparameters and reproducibility via configurations folder.
a. Main Components
1. Configuration Manager
• Reads .conf files, supplies hyperparameters and file paths
2. Data Ingestion & Preprocessing
• Raw data loader (CSV/TXT reader)
• Graph building (adjacency matrices, temporal slicing)
• Output: preprocessed NPZ datasets
3. Model Module
• ASTGNN class (graph convolution + temporal modeling layers)
4. Training Engine
• Optimizer, loss functions (from lib/utils & lib/metrics), checkpointing
5. Evaluation & Logging
• Metrics computation (MAE, RMSE, MAPE)
• Log outputs to console/files (nohup .out)
6. External Libraries
• PyTorch, NumPy, SciPy (for sparse graphs)
b. Relationships and Data Flow
c. Architectural Patterns
• Pipeline pattern: clear stages (prepare → train → evaluate)
• Separation of concerns: config, data/preprocessing, model definition, utilities, training logic
d. Key Technologies
• Python scripts orchestrating pipeline
• PyTorch for tensor operations and NN training
• Custom modules in lib/ for metrics and utilities
• Configuration files for experiment reproducibility
Detailed Capture of Architectural Elements
• Utility functions (lib/utils.py) as a shared service used by both preprocessing and training.
• Config folder as a reusable component feeding both prepareData.py and train_ASTGNN.py.