How servicenow/n-beats works
Project Type and Purpose
• This is an ML research‐pipeline repo that implements and reproduces the N‑BEATS univariate time‑series forecasting algorithm in PyTorch.
• It provides end-to-end components: dataset download & loading, model definition, experiment orchestration via configurable scripts, result storage, summary scripts, and analysis notebooks.
High-Level Components
• CLI/Build Layer
– Makefile targets (init, dataset, build, run, notebook) drive Docker image build, data download, experiment creation, execution, and analysis.
– Dockerfile wraps all dependencies into a container.
• Configuration Layer
– Gin configuration files under experiments/* define hyperparameters and pipeline settings (generic vs. interpretable model).
• Experiment Orchestrator
– experiments/main.py and experiments/trainer.py read gin configs, instantiate data loaders, models, and training loops.
• Common Utilities
– common/ holds shared code: experiment scaffolding, HTTP utilities, metrics, samplers, settings, and PyTorch extensions (losses, ops, snapshot).
• Data Layer
– datasets/ modules implement dataset download and PyTorch-compatible loaders for electricity, M3, M4, tourism, traffic.
• Model Layer
– models/nbeats.py defines the neural architecture (stacked fully connected blocks).
• Storage Layer
– Local FS directories under storage/ (created by make commands) store:
· raw datasets
· experiment configurations and generated commands
· training logs, model snapshots, loss curves, and forecast outputs
• Analysis Layer
– summary/ scripts compute aggregated metrics from saved forecasts.
Component Interactions and Data Flow
1. User invokes Makefile targets → Docker container initialization
2. Makefile → downloads datasets into storage/datasets via datasets/main.py
3. Makefile build target + gin config → for each ensemble member, writes commands into storage/experiments/...
4. Makefile run target → executes each command:
– Trainer script reads gin files, pulls data from datasets/, uses common utilities, loads models/nbeats.py, trains on GPU/CPU
– Outputs saved snapshots, logs, forecasts into storage/experiments/...
5. Summary scripts ingest forecasts, compute metrics, write to storage/summary
6. Notebooks connect to storage/experiments and storage/summary for visualizations
7. (Optional) Cloud scheduler can replace Makefile run loop, submitting each command to a cluster