How thinklab-sjtu/crossformer works
Project Type and Purpose
• This is a PyTorch-based research codebase for multivariate time-series forecasting using a novel Transformer variant (“Crossformer”).
• Key features: DSW embedding, Two-Stage Attention (TSA), Hierarchical Encoder-Decoder (HED).
• Entry points: main_crossformer.py (training/validation), eval_crossformer.py (evaluation/prediction).
Top-Level Components
a. CLI & Orchestration Layer
• main_crossformer.py
• eval_crossformer.py
• **scripts/\*.sh** (experiment runners)
b. Data Layer
• datasets/: raw CSV time series
• data/data_loader.py: Dataset and DataLoader wrappers
c. Model Layer
• cross_models/
• cross_embed.py (DSW embedding)
• cross_encoder.py, cross_decoder.py (HED encoder/decoder)
• attn.py (attention mechanisms, including TSA)
• cross_former.py (model orchestration)
d. Experiment Layer
• cross_exp/
• exp_basic.py (basic training loop)
• exp_crossformer.py (Crossformer-specific training/validation logic)
e. Utility Layer
• utils/metrics.py (evaluation metrics)
• utils/tools.py (misc utilities: checkpointing, logging)
f. Artifacts & Outputs
• checkpoints/ (saved models)
• results/ (metrics, predictions)
g. External Dependencies
• Python 3.7, PyTorch, numpy, pandas, einops
Relationships and Data Flow
1. User invokes main_crossformer.py (or shell script) →
2. data/data_loader.py loads CSVs from datasets/ →
3. cross_exp/exp_crossformer.py uses DataLoader →
4. cross_models/cross_former.py constructs the network by calling embedding, attention, encoder, decoder modules →
5. Training loop runs forward/backward, emits metrics →
6. utils/metrics.py computes loss and evaluation scores →
7. utils/tools.py handles checkpoint saving to checkpoints/ →
8. At end, results/ folder is populated with predictions and metrics →
9. eval_crossformer.py can be called to reload a checkpoint and produce final forecasts.
Architectural Patterns & Design Principles
• Modular layering: clear separation of data ingestion, model definition, experiment orchestration, utilities.
• Pipeline pattern: sequential data flow from loader → model → evaluation → storage.
• Dependency injection: main scripts configure hyperparameters via CLI flags and pass them to experiment classes.
• Reusability: cross_exp abstracts the training loop so new models can plug in.
• Research-friendly: scripts folder for one-line reproduction of batch experiments.