How zhangchenhaobest/tfb_time_series_forecasting_methods works
Project type and purpose
• TFB is an open-source Python library and benchmarking framework for time-series forecasting research.
• It provides:
• A unified pipeline for data ingestion, model execution, and evaluation
• A registry of baseline forecasting methods
• Evaluation strategies (fixed/rolling forecasts) and metrics
• Reporting outputs (CSV and a Dash dashboard)
• Utility scripts and a Docker container for reproducible experiments
File structure
• Top-level artifacts:
• Dockerfile, requirements.txt / requirements-docker.txt → containerization & dependencies
• config/*.json → evaluation and forecasting configurations (horizon, window, strategy)
• scripts/ → shell wrappers invoking experiments by dataset & model
• characteristics_extractor/ → standalone module computing time-series characteristics
• docs/ → static figures, tutorials, API docs links
• ts_benchmark/ → core Python package
• ts_benchmark package submodules:
• data/ → data loaders, dataset definitions, server and pool abstractions
• baselines/ → implementations of forecasting methods organized by algorithm
• evaluation/ → metric computations, evaluation strategies (fixed, rolling)
• models/ → model loader and base class
• pipeline.py → orchestration of load→train→predict→evaluate
• recording.py → experiment logging
• report/ → CSV report and Dash web app for leaderboard and queries
• utils/ → common utilities, parallel backends, design patterns
README for additional insights
• The “Pipeline.png” figure shows:
CLI → Pipeline → DataLoader → Model → Strategy → Metrics → Report
• Emphasizes fairness (no drop-last), extensibility (add new baselines), reproducibility (scripts + Docker)
a. Main components
• CLI Entry Point / Scripts (shell wrappers)
• Configuration Loader (reads JSON configs)
• Characteristics Extractor (optional feature module)
• Pipeline Orchestrator (ts_benchmark/pipeline.py)
• Data Layer (ts_benchmark/data)
• Model Registry & Loader (ts_benchmark/models + baselines)
• Evaluation Strategies & Metrics (ts_benchmark/evaluation)
• Reporting Engine (CSV reports + Dash app)
• External Runtimes (PyTorch, Docker container, Browser for Dash)
b. Relationships & Data flows
1. User runs a script or CLI → passes config-path, model name, dataset names →
2. Pipeline loads configuration → invokes DataLoader to fetch data →
3. (Optionally) Characteristics Extractor processes raw series →
4. Model Loader instantiates a baseline or user model →
5. Pipeline runs training / forecasting with chosen Evaluation Strategy →
6. Evaluation computes Metrics →
7. Reporting writes CSV and pushes results to Dash memory →
8. Dash app serves leaderboard UI
9. Recording module logs experiment metadata
c. Architectural patterns & design principles
• Strategy pattern for evaluation (fixed vs. rolling forecast)
• Plugin / registry pattern for baseline methods
• Factory for model instantiation via string identifiers
• Separation of concerns: data ←→ model ←→ evaluation ←→ reporting
• Containerized deployment for reproducibility (Dockerfile)
d. Key technologies & libraries
• Python 3.8+
• PyTorch 2.4.1 (model backends)
• Ray / sequential backends for parallel execution (ts_benchmark/utils/parallel)
• Plotly Dash for interactive leaderboard (ts_benchmark/report/report_dash)
• JSON for configuration (config/*.json)
• Shell scripts for experiment orchestration (scripts/)
Elements
• Blue for core modules (ts_benchmark/*)
• Green for data-related (data loaders, characteristics)
• Orange for evaluation & metrics
• Purple for reporting (CSV & Dash)
• Grey for external runtimes and containers (Docker, PyTorch, Browser)