How thuml/autoformer works
The project is a research/engineering repository for long-term time series forecasting. Its primary purpose is to implement and evaluate the Autoformer model – a Transformer variant that uses series decomposition and an auto-correlation mechanism for forecasting. In addition to Autoformer, the repo includes implementations of similarly themed models such as Informer, Transformer, and Reformer, and it provides extensive scripts for running experiments on multiple real-world datasets (e.g., energy, traffic, economics, weather, disease).
Main Components
a. Data Processing:
• Directory: data_provider
• Role: Contains routines to load and preprocess time series data (data_factory.py, data_loader.py), ensuring that datasets are pre-processed and structured for experiments.
b. Model Building:
• Directories: models and layers
• Role:
• models: Implements different time series forecasting models (Autoformer, Informer, Reformer, Transformer).
• layers: Provides the foundational layers and building blocks (e.g., AutoCorrelation for the series-wise mechanism, embedding modules, self-attention families) that support the models.
c. Experimentation and Execution:
• Directory: exp
• Role: Houses modules that set up and run model experiments (exp_basic.py, exp_main.py).
d. Script-based Experimentation:
• Directory: scripts (with subdirectories such as ETT_script, ECL_script, etc.)
• Role: Contains shell scripts that drive training experiments for various benchmarks across different domains (traffic, weather, exchange, etc.).
e. Utilities:
• Directory: utils
• Role: Helper functions for metrics, time features, data downloading, masking, and general tools to support other components.
f. Infrastructure and Deployment:
• Files: Dockerfile, Makefile, environment.yml, requirements.txt, run.py
• Role: Provide configuration for containerized execution (Docker), dependency management, reproducibility (Makefile commands), and a unified entry-point for running experiments.
Interactions and Relationships
• Data Flow:
• Data is then fed into the experimental workflow in exp/ (or triggered via scripts) which calls run.py to orchestrate model training and evaluation.
• Model Pipeline:
• Within the training/inference flow, models (from the models directory) are assembled using core layers (from layers) that encapsulate essential components such as the auto-correlation mechanism.
• Utilities provide supporting functionalities (e.g., metrics for evaluation, time features extraction) that are used during training and evaluation.
• Deployment:
• A Docker container (defined in Dockerfile), conda environment (environment.yml), and Makefile tasks ensure that the experiment scripts can be reliably executed in different environments.
• Script Execution:
• Multiple bash scripts under scripts/ directories allow the model to be run over different datasets/applications. They initiate the experiment process and ensure appropriate configuration is applied.
Architectural Patterns and Design Principles
• Modularity: The separation into data handling, model layers, experiment scripts, and utilities exemplifies a modular design.
• Experimentation-driven Design: The exp and scripts directories indicate a design geared towards conducting and reproducing experiments, reflecting the research focus.
• Container-based Deployment: Dockerfile and Makefile ensure the environment can be reproduced and experiments reliably executed, emphasizing reproducibility and deployment best practices.