How xiongxiaoxu/sst works
• Data Handling and Preprocessing:
• Modules: data_provider (data_factory.py, data_loader.py).
• Role: Ingest and prepare time series datasets.
• Experimentation and Execution Workflow:
• Modules: exp (exp_basic.py, exp_main.py) and scripts (e.g., electricity.sh, weather.sh, etc.).
• Role: Scripted entry-points and orchestrated experiment setups to run forecasting experiments.
• Model Architecture:
• Directories: models and layers.
• In “models”, you see multiple variants (e.g., AttMam, DecoderOnly, Mamba, MambaFormer, SST in one version).
• In “layers”, you see transformer-related modules (Embed.py, Mamba_Family.py, SelfAttention_Family.py, Transformer_EncDec.py, LWT_layers, Long_encoder, Short_encoder, RevIN) that build up various parts of the network.
• Role: Implement the core forecasting algorithms. Note that the design integrates a hybrid “Mamba-Transformer” concept – using global experts (Mamba) for long-range patterns and local experts (LWT) for shorter-term variation.
• Utility Functions:
• Modules: utils (masking.py, metrics.py, timefeatures.py, tools.py).
• Role: Provide support functions like feature processing, metric calculations, and data masking.
• (Optionally) Project Metadata and External Integration:
• README files, configuration of external packages (e.g., mamba-ssm package, PyTorch, and dependencies as listed).
• Role: Reference for external dependencies and overall system context.
Relationships and Data Flow
• Start at the Entry Points:
• Scripts and run.py/run_exp.py trigger the experiments.
• Experiment Module Coordination:
• The experiment modules coordinate the workflow: they call the Data Provider to load and configure input datasets, then build and run the models.
• Model and Layer Interactions:
• Inside the Models component, show that the individual models make extensive use of sub-components from the “layers” directory.
• Sub-blocks for “Mamba-Family”, “SelfAttention-Family”, “Embed”, “LWT_layers,” etc.
• Utility Integration:
• “utils” to both “models” and “exp” to show that metrics computation, masking, and additional feature processing are applied during training and evaluation.
• External Libraries and Dependencies:
• External dependencies (e.g., mamba-ssm, PyTorch, transformers) as a separate block that “models” and “layers” utilize.
Architectural Patterns & Design Principles
• Separation of Concerns:
• The structure clearly divides responsibilities (data handling, experiment coordination, model construction, and utility functions).
• Modular Design:
• That modules from “layers” and “models” can be exchanged or extended easily (reflecting the plug-and-play design for different forecasting experiments).
• Hybrid Modeling Approach:
• Scalability:
• Linear complexity feature O(L) mentioned in the README, which emphasizes the scalability in time series length.
Technology and Implementation Details
• Annotations for key frameworks and dependencies (Python 3.10, PyTorch 1.12.1+cu116, mamba-ssm, numpy, transformers).
• Data flow from raw inputs—loaded via data_provider—to processed features, subsequently fed to the multiple model layers.
• Data Provider (including its two files: data_loader and data_factory).
• Experiment Runner (exp modules and scripts).
• Model Architectures (models folder connected to a subset panel showing the layers that construct them—Mamba, SelfAttention, LWT, etc.).
• Utils and External Dependencies.
Happy diagramming!