# thinklab-sjtu/crossformer architecture

> Architecture diagram of the GitHub repository thinklab-sjtu/crossformer, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-09-24). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/thinklab-sjtu/crossformer
- Repository: https://github.com/thinklab-sjtu/crossformer

## Overview

##### Project Type and Purpose
- This is a PyTorch-based research codebase for multivariate time-series forecasting using a novel Transformer variant (“Crossformer”).
- Key features: DSW embedding, Two-Stage Attention (TSA), Hierarchical Encoder-Decoder (HED).
- Entry points: `main_crossformer.py` (training/validation), `eval_crossformer.py` (evaluation/prediction).

##### Top-Level Components
a. CLI & Orchestration Layer
- **main_crossformer.py**
- **eval_crossformer.py**
- **scripts/\*.sh** (experiment runners)
b. Data Layer
- **datasets/**: raw CSV time series
- **data/data_loader.py**: Dataset and DataLoader wrappers
c. Model Layer
- **cross_models/**
- cross_embed.py (DSW embedding)
- cross_encoder.py, cross_decoder.py (HED encoder/decoder)
- attn.py (attention mechanisms, including TSA)
- cross_former.py (model orchestration)
d. Experiment Layer
- **cross_exp/**
- exp_basic.py (basic training loop)
- exp_crossformer.py (Crossformer-specific training/validation logic)
e. Utility Layer
- **utils/metrics.py** (evaluation metrics)
- **utils/tools.py** (misc utilities: checkpointing, logging)
f. Artifacts & Outputs
- **checkpoints/** (saved models)
- **results/** (metrics, predictions)
g. External Dependencies
- Python 3.7, PyTorch, numpy, pandas, einops

##### Relationships and Data Flow
1. User invokes **main_crossformer.py** (or shell script) →
2. **data/data_loader.py** loads CSVs from **datasets/** →
3. **cross_exp/exp_crossformer.py** uses DataLoader →
4. **cross_models/cross_former.py** constructs the network by calling embedding, attention, encoder, decoder modules →
5. Training loop runs forward/backward, emits metrics →
6. **utils/metrics.py** computes loss and evaluation scores →
7. **utils/tools.py** handles checkpoint saving to **checkpoints/** →
8. At end, **results/** folder is populated with predictions and metrics →
9. **eval_crossformer.py** can be called to reload a checkpoint and produce final forecasts.

##### Architectural Patterns & Design Principles
- Modular layering: clear separation of data ingestion, model definition, experiment orchestration, utilities.
- Pipeline pattern: sequential data flow from loader → model → evaluation → storage.
- Dependency injection: main scripts configure hyperparameters via CLI flags and pass them to experiment classes.
- Reusability: `cross_exp` abstracts the training loop so new models can plug in.
- Research-friendly: scripts folder for one-line reproduction of batch experiments.

## Diagram (Mermaid)

```mermaid
flowchart TD
    %% CLI Layer
    subgraph "CLI & Orchestration Layer"
        CLI_main["main_crossformer.py"]:::cli
        CLI_eval["eval_crossformer.py"]:::cli
        Scripts["scripts/*.sh"]:::cli
    end

    %% Data Layer
    subgraph "Data Layer"
        Datasets["datasets/"]:::data
        DataLoader["data/data_loader.py"]:::data
    end

    %% Experiment Layer
    subgraph "Experiment Layer"
        ExpBasic["cross_exp/exp_basic.py"]:::exp
        ExpCF["cross_exp/exp_crossformer.py"]:::exp
    end

    %% Model Layer
    subgraph "Model Layer"
        Embed["DSW Embedding\n(cross_embed.py)"]:::model
        Attn["TSA Attention\n(attn.py)"]:::model
        Enc["HED Encoder\n(cross_encoder.py)"]:::model
        Dec["HED Decoder\n(cross_decoder.py)"]:::model
        Orchestrator["Crossformer Orchestrator\n(cross_former.py)"]:::model
    end

    %% Utility Layer
    subgraph "Utility Layer"
        Metrics["utils/metrics.py"]:::util
        Tools["utils/tools.py"]:::util
    end

    %% Artifacts & Outputs
    subgraph "Artifacts & Outputs"
        Checkpoints["checkpoints/"]:::artifact
        Results["results/"]:::artifact
    end

    %% External Dependencies
    subgraph "External Dependencies"
        ExtDep["Python3.7, PyTorch, numpy,\npandas, einops"]:::external
    end

    %% Relationships
    Scripts -->|runs| CLI_main
    CLI_main -->|loads data| DataLoader
    DataLoader -->|provides batches| ExpCF
    ExpCF -->|uses training loop| ExpBasic
    ExpCF -->|constructs model| Orchestrator
    Orchestrator -->|uses embedding| Embed
    Orchestrator -->|uses attention| Attn
    Orchestrator -->|uses encoder| Enc
    Orchestrator -->|uses decoder| Dec
    Orchestrator -->|emits metrics| Metrics
    Metrics -->|computes scores| Tools
    Tools -->|saves checkpoints| Checkpoints
    Tools -->|writes outputs| Results

    CLI_eval -->|reloads checkpoint| Tools
    CLI_eval -->|generates forecasts| Results

    ExtDep --> CLI_main
    ExtDep --> DataLoader
    ExtDep --> ExpCF
    ExtDep --> Orchestrator
    ExtDep --> Metrics

    %% Click Events
    click CLI_main "https://github.com/thinklab-sjtu/crossformer/blob/master/main_crossformer.py"
    click CLI_eval "https://github.com/thinklab-sjtu/crossformer/blob/master/eval_crossformer.py"
    click Scripts "https://github.com/thinklab-sjtu/crossformer/tree/master/scripts/"
    click Datasets "https://github.com/thinklab-sjtu/crossformer/tree/master/datasets/"
    click DataLoader "https://github.com/thinklab-sjtu/crossformer/blob/master/data/data_loader.py"
    click ExpBasic "https://github.com/thinklab-sjtu/crossformer/blob/master/cross_exp/exp_basic.py"
    click ExpCF "https://github.com/thinklab-sjtu/crossformer/blob/master/cross_exp/exp_crossformer.py"
    click Embed "https://github.com/thinklab-sjtu/crossformer/blob/master/cross_models/cross_embed.py"
    click Attn "https://github.com/thinklab-sjtu/crossformer/blob/master/cross_models/attn.py"
    click Enc "https://github.com/thinklab-sjtu/crossformer/blob/master/cross_models/cross_encoder.py"
    click Dec "https://github.com/thinklab-sjtu/crossformer/blob/master/cross_models/cross_decoder.py"
    click Orchestrator "https://github.com/thinklab-sjtu/crossformer/blob/master/cross_models/cross_former.py"
    click Metrics "https://github.com/thinklab-sjtu/crossformer/blob/master/utils/metrics.py"
    click Tools "https://github.com/thinklab-sjtu/crossformer/blob/master/utils/tools.py"

    %% Styles
    classDef cli fill:#f9f,stroke:#333,stroke-width:1px;
    classDef data fill:#ccf,stroke:#333,stroke-width:1px;
    classDef exp fill:#f96,stroke:#333,stroke-width:1px;
    classDef model fill:#9f9,stroke:#333,stroke-width:1px;
    classDef util fill:#ddd,stroke:#333,stroke-width:1px;
    classDef artifact fill:#ffc,stroke:#333,stroke-width:1px;
    classDef external fill:#eef,stroke:#333,stroke-width:1px,stroke-dasharray: 5 5;
```
