# thuml/itransformer architecture

> Architecture diagram of the GitHub repository thuml/itransformer, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-03-30). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/thuml/itransformer
- Repository: https://github.com/thuml/itransformer

## Overview

##### Main Components
- Data Provider: Represented by the “data_provider” directory (data_factory.py and data_loader.py). This component is responsible for preparing, loading, and possibly preprocessing the time series data.
- Core Model & Layers: Split into two parts:
- Model Implementations in the “model” directory (e.g., Flashformer.py, Flowformer.py, Informer.py, etc. and their inverted counterparts). These are the various forecasting model definitions.
- Lower-level Transformer Layers located in the “layers” directory (e.g., Embed.py, SelfAttention_Family.py, Transformer_EncDec.py) which implement the core building blocks or modules of the Transformer architectures.
- Experimentation Framework: Represented by the “experiments” directory (exp_basic.py, exp_long_term_forecasting.py, etc.), which shows how different configurations or experiments are set up to test the forecasting models.
- Scripts for Reproducibility & Performance Boosting: The “scripts” directory is heavily organized into subfolders for various experiment runs:
- Performance boosting (e.g., “boost_performance”) for comparing different models,
- Lookback increasing (e.g., “increasing_lookback”),
- Multivariate forecasting, variate generalization, and model efficiency.
- Utility Modules: The “utils” directory provides helper functions (masking, metrics, timefeatures, etc.) that assist various components of the system.
- Main Execution Entry: The “run.py” file, which likely serves as a high-level entry point tying some of these components together.

##### Relationships and Interactions
- The Data Provider component supplies data to the Core Models.
- The Core Models use the Layers (i.e., specific implementations of attention and embedding modules) and are integrated with utility functions.
- Experiment scripts (and run.py) orchestrate the training, evaluation, and comparison of these models based on provided datasets.
- Scripts in subdirectories further encapsulate run configurations (for different performance boosting scenarios or forecasting setups) by invoking the models with particular settings and datasets.
- Dependencies flow from high-level experiment execution (scripts) down to core modeling (model and layers) and then to data handling and utilities.

##### Architectural Patterns and Design Principles
- Modularity: The project cleanly separates data ingestion (data_provider), model building (model and layers), and evaluation/experimentation (experiments and scripts).
- Reusability: The existence of utility and helper functions (utils) and modular design in defining different Transformer variants (both standard and inverted).
- Experiment-Driven: The scripts and experiments directories indicate that the architectural design focuses on reproducibility and comparative analysis following research best practices.
- Clear Separation of Concerns: Input/output handling, core model definitions, and experimental validations are all compartmentalized.

##### Relevant Technologies, Frameworks, and Libraries
- PyTorch: As inferred from the README instructions (“pip install -r requirements.txt”) and typical usage in Transformer implementations.
- External Codebases: References to FlashAttention, and inspiration from projects like Reformer, Informer, Autoformer, etc.
- Shell Scripting: Scripts for executing experiments in Linux environments (e.g., bash scripts in the “scripts” folders).

## Diagram (Mermaid)

```mermaid
graph TD
    subgraph "Data Ingestion"
        DF["Data Factory"]:::data
        DL["Data Loader"]:::data
    end

    subgraph "Core Models"
        MI["Model Implementations"]:::model
        TL["Transformer Layers"]:::layers
    end

    U["Utility Modules"]:::util

    subgraph "Experiment Runner"
        EF["Experiment Framework"]:::experiment
        SE["Scripts for Execution"]:::scripts
        R["Main Execution (run.py)"]:::execution
    end

    DF -->|"flows_to"| MI
    DF -->|"flows_to"| TL
    DL -->|"flows_to"| MI
    DL -->|"flows_to"| TL
    MI -->|"utilizes"| U
    TL -->|"utilizes"| U
    MI -->|"integrated_in"| EF
    TL -->|"integrated_in"| EF
    U -->|"supports"| EF
    U -->|"supports"| SE
    EF -->|"orchestrates"| R
    SE -->|"orchestrates"| R

    click DF "https://github.com/thuml/itransformer/blob/main/data_provider/data_factory.py"
    click DL "https://github.com/thuml/itransformer/blob/main/data_provider/data_loader.py"
    click MI "https://github.com/thuml/itransformer/tree/main/model/"
    click TL "https://github.com/thuml/itransformer/tree/main/layers/"
    click EF "https://github.com/thuml/itransformer/tree/main/experiments/"
    click SE "https://github.com/thuml/itransformer/tree/main/scripts/"
    click U "https://github.com/thuml/itransformer/tree/main/utils/"
    click R "https://github.com/thuml/itransformer/blob/main/run.py"

    classDef data fill:#f9c,stroke:#333,stroke-width:2px;
    classDef model fill:#cff,stroke:#333,stroke-width:2px;
    classDef layers fill:#cfc,stroke:#333,stroke-width:2px;
    classDef util fill:#fc9,stroke:#333,stroke-width:2px;
    classDef experiment fill:#ccf,stroke:#333,stroke-width:2px;
    classDef scripts fill:#ffc,stroke:#333,stroke-width:2px;
    classDef execution fill:#c9f,stroke:#333,stroke-width:2px;
```
