# panzijie825/s2ip-llm architecture

> Architecture diagram of the GitHub repository panzijie825/s2ip-llm, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-05-04). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/panzijie825/s2ip-llm
- Repository: https://github.com/panzijie825/s2ip-llm

## Overview

##### Project Type and Purpose
- This is a research-oriented machine-learning pipeline for time-series forecasting.
- It provides both Long-term and Short-term forecasting modules, each ingesting data, preprocessing it, learning prompts for an LLM, and evaluating predictions.

##### File Structure
- Top-level folders:
- Long-term_Forecasting
- Short-term_Forecasting
- Each forecasting folder contains:
- data_provider (data ingestion & loading)
- models (S2IPLLM model and prompt logic)
- utils (preprocessing, metric calculation, tokenization)
- scripts (shell scripts to launch experiments for different datasets)
- runner scripts (run.py / run_m4.py)
- exp folder in Long-term (experiment orchestration code)
- A shared requirements.txt and environment setup; external datasets download from Google Drive.

##### README for Insights
- Dependencies managed via Conda and pip.
- Data must be placed under `./data/`.
- Execution is via shell scripts that invoke Python runners.
- No web UI or microservice boundaries—this is a batch ML pipeline.

##### Main Components & Their Interactions
- Data Ingestion
- External Data Source (Google Drive) → Local `./data/`
- data_provider/data_loader → pandas or PyTorch Datasets
- Preprocessing & Feature Engineering
- utils/timefeatures, masking, tokenization → transformed tensors
- Model & Prompt Module
- models/S2IPLLM.py (core LLM interface)
- models/prompt.py (prompt construction logic)
- Experiment Runner
- Long-term: exp_basic.py, exp_long_term_forecasting.py → orchestrate training loops, logging
- Short-term: run_m4.py → similar orchestration for M4 data
- Script Launcher
- scripts/*.sh → pass dataset-specific parameters into runners
- Evaluation & Metrics
- utils/metrics.py, m4_summary.py, losses.py → compute performance, save logs
- Environment & Dependencies
- Conda environment, requirements.txt

##### Relationships & Data Flow
- Feedback loops if any (e.g., hyperparameter tuning or multi-stage experiments in exp/).

##### Architectural Patterns & Design Principles
- Modularity: clear separation between data ingestion, preprocessing, modeling, and evaluation.
- Reusability: shared utils across both forecasting modules.
- Experiment Repeatability: shell scripts + runner scripts enforce reproducible runs.
- Layered pipeline: Data → Feature Engine → Model → Evaluation.

##### Technologies & Frameworks
- Python 3.10
- Conda + pip
- PyTorch or TensorFlow (implied by LLM usage)
- Large Language Model API or library (e.g., Hugging Face Transformers)
- Shell for scripting experiments
- pandas, NumPy for data handling

## Diagram (Mermaid)

```mermaid
flowchart TD
    subgraph "Long-term Forecasting Pipeline"
        direction TB
        LT_Scripts["Shell Scripts"]:::scripts
        LT_Scripts --> LT_Data_Factory["data_factory.py"]:::data
        LT_Data_Factory --> LT_Data_Loader["data_loader.py"]:::data
        LT_Data_Loader --> LT_Pre_Time["timefeatures.py"]:::data
        LT_Pre_Time --> LT_Pre_Token["tokenization.py"]:::data
        LT_Pre_Token --> LT_Pre_Tools["tools.py"]:::data
        LT_Pre_Tools --> LT_Model_Core["S2IPLLM.py"]:::model
        LT_Model_Core --> LT_Model_Prompt["prompt.py"]:::model
        LT_Model_Prompt --> LT_Eval["Eval & Metrics"]:::utils
    end

    subgraph "Short-term Forecasting Pipeline"
        direction TB
        ST_Scripts["m4.sh"]:::scripts
        ST_Scripts --> ST_Data_Factory["data_factory.py"]:::data
        ST_Data_Factory --> ST_Data_Loader["data_loader.py"]:::data
        ST_Data_Loader --> ST_Data_M4["m4.py"]:::data
        ST_Data_M4 --> ST_Pre_Time["timefeatures.py"]:::data
        ST_Pre_Time --> ST_Pre_Mask["masking.py"]:::data
        ST_Pre_Mask --> ST_Pre_Tools["tools.py"]:::data
        ST_Pre_Tools --> ST_Model_Core["S2IPLLM.py"]:::model
        ST_Model_Core --> ST_Model_Prompt["prompt.py"]:::model
        ST_Model_Prompt --> ST_Eval["Eval & Metrics"]:::utils
    end

    Env["requirements.txt"]:::env

    click LT_Data_Factory "https://github.com/panzijie825/s2ip-llm/blob/main/Long-term_Forecasting/data_provider/data_factory.py"
    click LT_Data_Loader "https://github.com/panzijie825/s2ip-llm/blob/main/Long-term_Forecasting/data_provider/data_loader.py"
    click ST_Data_Factory "https://github.com/panzijie825/s2ip-llm/blob/main/Short-term_Forecasting/data_provider/data_factory.py"
    click ST_Data_Loader "https://github.com/panzijie825/s2ip-llm/blob/main/Short-term_Forecasting/data_provider/data_loader.py"
    click ST_Data_M4 "https://github.com/panzijie825/s2ip-llm/blob/main/Short-term_Forecasting/data_provider/m4.py"
    click LT_Pre_Time "https://github.com/panzijie825/s2ip-llm/blob/main/Long-term_Forecasting/utils/timefeatures.py"
    click LT_Pre_Token "https://github.com/panzijie825/s2ip-llm/blob/main/Long-term_Forecasting/utils/tokenization.py"
    click LT_Pre_Tools "https://github.com/panzijie825/s2ip-llm/blob/main/Long-term_Forecasting/utils/tools.py"
    click ST_Pre_Time "https://github.com/panzijie825/s2ip-llm/blob/main/Short-term_Forecasting/utils/timefeatures.py"
    click ST_Pre_Mask "https://github.com/panzijie825/s2ip-llm/blob/main/Short-term_Forecasting/utils/masking.py"
    click ST_Pre_Tools "https://github.com/panzijie825/s2ip-llm/blob/main/Short-term_Forecasting/utils/tools.py"
    click LT_Model_Core "https://github.com/panzijie825/s2ip-llm/blob/main/Long-term_Forecasting/models/S2IPLLM.py"
    click LT_Model_Prompt "https://github.com/panzijie825/s2ip-llm/blob/main/Long-term_Forecasting/models/prompt.py"
    click ST_Model_Core "https://github.com/panzijie825/s2ip-llm/blob/main/Short-term_Forecasting/models/S2IPLLM.py"
    click ST_Model_Prompt "https://github.com/panzijie825/s2ip-llm/blob/main/Short-term_Forecasting/models/prompt.py"
    click LT_Eval "https://github.com/panzijie825/s2ip-llm/tree/main/Long-term_Forecasting/utils"
    click ST_Eval "https://github.com/panzijie825/s2ip-llm/tree/main/Short-term_Forecasting/utils"
    click LT_Scripts "https://github.com/panzijie825/s2ip-llm/tree/main/Long-term_Forecasting/scripts"
    click ST_Scripts "https://github.com/panzijie825/s2ip-llm/blob/main/Short-term_Forecasting/m4.sh"
    click Env "https://github.com/panzijie825/s2ip-llm/blob/main/requirements.txt"

    classDef data fill:#BBDEFB,stroke:#0D47A1,color:#0D47A1
    classDef model fill:#C8E6C9,stroke:#1B5E20,color:#1B5E20
    classDef scripts fill:#FFE0B2,stroke:#E65100,color:#E65100
    classDef utils fill:#CFD8DC,stroke:#37474F,color:#37474F
    classDef env fill:#F5F5F5,stroke:#9E9E9E,color:#424242
```
