How rajatsen91/deepglo works
1. Repository purpose
- This is a research codebase for reproducing the DeepGLO paper’s forecasting experiments, not a general-purpose application or service. The main concern is training and evaluating time-series models on benchmark datasets.
2. Top-level structure
- The repo splits into three practical areas:
- DeepGLO/: reusable model and support library
- run_scripts/: dataset-specific experiment entrypoints
- datasets/: data acquisition and reshaping utilities
3. Core model orchestration
- DeepGLO/DeepGLO.py is the central coordination layer. It likely exposes the main model class that wires together preprocessing, the local learner, and the broader forecasting logic.
- This is the main boundary for training and prediction: scripts call into this module rather than constructing components directly.
4. Local temporal model
- DeepGLO/LocalModel.py implements the TCN-based “LeveledInit” local model mentioned in the README.
- Architecturally, this is the short-horizon sequence learner that handles temporal pattern extraction on the raw or normalized series.
5. Global/auxiliary modeling
- DeepGLO/Ftree.py appears to support the global side of the method, likely a factor/tree-style structure used by the overall DeepGLO algorithm.
- In the architecture diagram, treat this as the complementary model component that operates across series or captures shared structure beyond the local TCN.
6. Data loading and preprocessing
- DeepGLO/data_loader.py is the ingestion boundary between dataset files and model-ready tensors/arrays.
- DeepGLO/time.py likely handles time indexing, windowing, or calendar-related transformations.
- DeepGLO/utilities.py contains shared helpers used across data preparation, training, and evaluation.
7. Evaluation layer
- DeepGLO/metrics.py centralizes forecast scoring, which is important because the repo is designed for reproducing paper results.
- The evaluation flow is typically: model predictions → metrics module → reported results.
8. Experiment entrypoints
- run_scripts/run_electricity.py, run_pems.py, run_traffic.py, and run_wiki.py are dataset-specific drivers.
- These scripts are the user-facing orchestration layer: they load a dataset, configure normalization and model settings, run training/inference, and print or save results.
9. Dataset preparation boundary
- datasets/download-data.sh fetches the raw benchmark data.
- datasets/reshape_data.py converts downloaded data into the format expected by the loaders and experiment scripts.
- This means the repo has a clear offline preprocessing step before model training begins.
10. Data flow
- The main flow is: raw benchmark data → download/reshape scripts → loaders/time utilities → DeepGLO model wrapper → LocalModel plus global component → metrics.
- The run scripts sit at the top of this pipeline and choose dataset-specific parameters and normalization mode.
11. Runtime and dependencies
- The code expects PyTorch with CUDA for training, which makes GPU execution a core architectural assumption.
- Supporting packages (numpy, scipy, pandas, scikit-learn, matplotlib) indicate a scientific Python pipeline with preprocessing, evaluation, and result visualization.
12. How to diagram it
- A good architecture diagram would show:
- dataset preparation on the left,
- the run_scripts layer as the entrypoint,
- DeepGLO as the central orchestration package,
- LocalModel and the global component as internal model branches,
- and metrics as the output/evaluation sink.
- This is a single-machine research pipeline rather than a distributed or service-oriented system.
Main entry points: DeepGLO.py DeepGLO core · data_loader.py Data loader · __init__.py Package init · run_electricity.py Electricity run
Components 14
Dataset prep
Offline acquisition and reshaping of benchmark time-series data.
- Download data · Fetches raw benchmark datasets before experimentation.
datasets/download-data.sh - Reshape data · Converts downloaded data into loader-friendly experiment format.
datasets/reshape_data.py
Experiments
Dataset-specific entrypoints that configure and launch runs.
- Electricity run · Entrypoint for the electricity forecasting benchmark.
run_scripts/run_electricity.py - PEMS run · Entrypoint for the PEMS forecasting benchmark.
run_scripts/run_pems.py - Traffic run · Entrypoint for the traffic forecasting benchmark.
run_scripts/run_traffic.py - Wiki run · Entrypoint for the Wikipedia forecasting benchmark.
run_scripts/run_wiki.py
DeepGLO package
Reusable model, loading, timing, and evaluation code.
- DeepGLO core · Main coordination layer that wires preprocessing, local modeling, and overall forecasting logic.
DeepGLO/DeepGLO.py - Data loader · Loads prepared datasets into model-ready arrays and tensors.
DeepGLO/data_loader.py - Time utils · Handles windowing, time indexing, or calendar-related transformations.
DeepGLO/time.py - Shared utils · Common helper functions used across training, preprocessing, and evaluation.
DeepGLO/utilities.py - Local model · LeveledInit temporal learner for short-horizon pattern extraction.
DeepGLO/LocalModel.py - Global model · Complementary component that captures cross-series structure in DeepGLO.
DeepGLO/Ftree.py - Metrics · Centralizes forecast scoring and result reporting.
DeepGLO/metrics.py - Package init · Package surface for importing DeepGLO components.
DeepGLO/__init__.py
Connections 20
- Download data → Reshape data: raw data · Downloaded benchmark files are reshaped before use.
- Reshape data → Data loader: prepared files · Reshaped datasets are consumed by the loader boundary.
- Electricity run → DeepGLO core: configure/train · Dataset script instantiates and runs the model pipeline.
- PEMS run → DeepGLO core: configure/train · Dataset script instantiates and runs the model pipeline.
- Traffic run → DeepGLO core: configure/train · Dataset script instantiates and runs the model pipeline.
- Wiki run → DeepGLO core: configure/train · Dataset script instantiates and runs the model pipeline.
- Electricity run → Data loader: load data · Entrypoints use the ingestion layer to obtain model inputs.
- PEMS run → Data loader: load data · Entrypoints use the ingestion layer to obtain model inputs.
- Traffic run → Data loader: load data · Entrypoints use the ingestion layer to obtain model inputs.
- Wiki run → Data loader: load data · Entrypoints use the ingestion layer to obtain model inputs.
- DeepGLO core → Local model: local branch · Core wrapper delegates short-horizon learning to the TCN-based local model.
- DeepGLO core → Global model: global branch · Core wrapper combines the complementary cross-series component.
- DeepGLO core → Time utils: time features · Wrapper uses time indexing and windowing utilities.
- DeepGLO core → Shared utils: shared helpers · Wrapper relies on common helper functions.
- DeepGLO core → Metrics: evaluate · Predictions are scored through the metrics layer.
- Data loader → Time utils: index windows · Loader depends on time utilities for sequence shaping.
- Data loader → Shared utils: prep helpers · Loader shares helper functions for preprocessing.
- Package init → DeepGLO core: exports · Package init exposes the main model API.
- Package init → Local model: exports · Package init exposes the local model component.
- Package init → Global model: exports · Package init exposes the global component.