# facebookresearch/spinquant architecture

> Architecture diagram of the GitHub repository facebookresearch/spinquant, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-04-06). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/facebookresearch/spinquant
- Repository: https://github.com/facebookresearch/spinquant

## Overview

##### Main Components
- Model Input & Preprocessing – Starting from user-specified model paths and quantization parameters.
- Rotation Optimization Module – This includes scripts and methods for computing optimized rotation matrices. Look at the files in train_utils (e.g., apply_r3_r4.py, fsdp_trainer.py, optimizer.py, quant_linear.py) that focus on modifying or optimizing the model’s weights via rotations.
- Evaluation Module – Encompassing files in eval_utils (e.g., gptq_utils.py, modeling_llama.py, rotation_utils.py) to evaluate performance post quantization. This module also covers different schemes (such as GPTQ, RTN) based on the chosen settings.
- Utility and Helper Functions – A collection of support scripts in the utils folder (e.g., process_args.py, model_utils.py, quant_utils.py) that handle data conversion, argument processing, and specialized computations like Hadamard transforms.
- Script Wrappers and Execution Pipelines – The scripts directory contains various bash scripts that orchestrate the flow (e.g., optimizing rotation, evaluating PTQ, and exporting for ExecuTorch). These tie the modules together and determine the runtime steps.

##### Depicting the Relationships and Data Flow
- A branch from both the optimization and evaluation modules to an “Export Module” for ExecuTorch (this shows the conversion of results into a runtime-efficient, quantized model).
- Utility layers (helpers and configuration) surrounding these core components, highlighting how they support argument parsing, file conversions, and additional model utilities.

##### Noting Architectural Patterns and Principles
- Separation of concerns is applied here: optimization, evaluation, and export functionalities are segmented into different modules (each with their own directory).
- Script-based orchestration – the bash scripts provide high-level workflow commands, indicating a pipeline design.

##### Specific Elements
- Each Module Block:
- “Input & Configuration”: captures user parameters, command-line arguments, and initial settings.
- “Rotation Optimization Module”: shows files such as train_utils/apply_r3_r4.py, fsdp_trainer.py, and optimizer.py.
- “Evaluation Module”: illustrates eval_utils components (gptq_utils.py, modeling_llama.py, rotation_utils.py) along with PTQ evaluation scripts.
- “Export/Integration Module”: depicts the conversion to ExecuTorch and integration with external quantization kernels.
- “Utility & Helper Functions”: represents the utils directory’s functions (data processing, monkeypatching, etc.).
- External Dependencies Block:
- Libraries: PyTorch (>=2.0 with CUDA), fast-hadamard-transform, Hugging Face Transformers, etc.
- External model repositories and links to ExecuTorch documentation.

## Diagram (Mermaid)

```mermaid
flowchart TD
    %% Subgraph: Input & Configuration
    subgraph "Input & Configuration"
        Args["Input Parameters (process_args.py)"]:::input
        Doc["Documentation (README.md)"]:::input
    end

    %% Subgraph: Rotation Optimization Module (train_utils)
    subgraph "Rotation Optimization Module (train_utils)"
        rot1["apply_r3_r4.py"]:::rotation
        rot2["fsdp_trainer.py"]:::rotation
        rot3["optimizer.py"]:::rotation
        rot4["quant_linear.py"]:::rotation
    end

    %% Node representing the output of rotation optimization
    OptMatrix["Optimized Rotation Matrix"]:::rotation

    %% Subgraph: Evaluation Module (eval_utils)
    subgraph "Evaluation Module (eval_utils)"
        eval1["gptq_utils.py"]:::evaluation
        eval2["modeling_llama.py"]:::evaluation
        eval3["rotation_utils.py"]:::evaluation
        eval4["ptq.py"]:::evaluation
    end

    %% Node representing the final quantized model after evaluation
    Quantized["Quantized Model"]:::evaluation

    %% Subgraph: Export/Integration Module
    subgraph "Export/Integration Module"
        exp1["convert_to_executorch.py"]:::export
        exp2["31_optimize_rotation_executorch.sh"]:::export
        exp3["32_eval_ptq_executorch.sh"]:::export
    end

    %% Subgraph: Utility & Helper Functions
    subgraph "Utility & Helper Functions"
        util1["model_utils.py"]:::utility
        util2["quant_utils.py"]:::utility
        util3["data_utils.py"]:::utility
        util4["hadamard_utils.py"]:::utility
        util5["monkeypatch.py"]:::utility
        util6["eval_utils.py"]:::utility
        util7["fuse_norm_utils.py"]:::utility
        util8["utils.py"]:::utility
    end

    %% Subgraph: Script Wrappers and Execution Pipelines
    subgraph "Script Wrappers and Execution Pipelines"
        scr1["10_optimize_rotation.sh"]:::script
        scr2["11_optimize_rotation_fsdp.sh"]:::script
        scr3["2_eval_ptq.sh"]:::script
        scr4["main.py (train_utils)"]:::script
        scr5["main.py (eval_utils)"]:::script
    end

    %% External Dependencies Block
    ext["External Dependencies<br/>(PyTorch, fast-hadamard-transform,<br/>Hugging Face, ExecuTorch)"]:::external

    %% Connections - Main Data Flow
    Args -->|"model_path&params"| rot1
    Doc --> rot1
    rot3 -->|"compute rotations"| OptMatrix
    OptMatrix -->|"apply optimized"| eval1
    eval4 -->|"evaluate PTQ"| Quantized
    Quantized -->|"export quantized"| exp1
    %% Additional branch from rotation optimization to export for direct integration
    OptMatrix -->|"export rotations"| exp1

    %% Support Connections from Utility & Script layers and External Dependencies
    util1 ---|"support"| rot1
    util1 ---|"support"| eval1
    util1 ---|"support"| exp1

    scr1 ---|"orchestrate"| rot1
    scr1 ---|"orchestrate"| eval1
    scr1 ---|"orchestrate"| exp1

    ext ---|"dependency"| rot1
    ext ---|"dependency"| eval1
    ext ---|"dependency"| exp1

    %% Styling Classes
    classDef input fill:#f9d5e5,stroke:#333,stroke-width:2px;
    classDef rotation fill:#ffd1a9,stroke:#333,stroke-width:2px;
    classDef evaluation fill:#a9d1ff,stroke:#333,stroke-width:2px;
    classDef export fill:#d1f9a9,stroke:#333,stroke-width:2px;
    classDef utility fill:#e0c3fc,stroke:#333,stroke-width:2px;
    classDef script fill:#c3fce0,stroke:#333,stroke-width:2px;
    classDef external fill:#fceaab,stroke:#333,stroke-width:2px;

    %% Click Events for Input & Configuration
    click Args "https://github.com/facebookresearch/spinquant/blob/main/utils/process_args.py"
    click Doc "https://github.com/facebookresearch/spinquant/blob/main/README.md"

    %% Click Events for Rotation Optimization Module
    click rot1 "https://github.com/facebookresearch/spinquant/blob/main/train_utils/apply_r3_r4.py"
    click rot2 "https://github.com/facebookresearch/spinquant/blob/main/train_utils/fsdp_trainer.py"
    click rot3 "https://github.com/facebookresearch/spinquant/blob/main/train_utils/optimizer.py"
    click rot4 "https://github.com/facebookresearch/spinquant/blob/main/train_utils/quant_linear.py"

    %% Click Events for Evaluation Module
    click eval1 "https://github.com/facebookresearch/spinquant/blob/main/eval_utils/gptq_utils.py"
    click eval2 "https://github.com/facebookresearch/spinquant/blob/main/eval_utils/modeling_llama.py"
    click eval3 "https://github.com/facebookresearch/spinquant/blob/main/eval_utils/rotation_utils.py"
    click eval4 "https://github.com/facebookresearch/spinquant/blob/main/ptq.py"

    %% Click Events for Export/Integration Module
    click exp1 "https://github.com/facebookresearch/spinquant/blob/main/utils/convert_to_executorch.py"
    click exp2 "https://github.com/facebookresearch/spinquant/blob/main/scripts/31_optimize_rotation_executorch.sh"
    click exp3 "https://github.com/facebookresearch/spinquant/blob/main/scripts/32_eval_ptq_executorch.sh"

    %% Click Events for Utility & Helper Functions
    click util1 "https://github.com/facebookresearch/spinquant/blob/main/utils/model_utils.py"
    click util2 "https://github.com/facebookresearch/spinquant/blob/main/utils/quant_utils.py"
    click util3 "https://github.com/facebookresearch/spinquant/blob/main/utils/data_utils.py"
    click util4 "https://github.com/facebookresearch/spinquant/blob/main/utils/hadamard_utils.py"
    click util5 "https://github.com/facebookresearch/spinquant/blob/main/utils/monkeypatch.py"
    click util6 "https://github.com/facebookresearch/spinquant/blob/main/utils/eval_utils.py"
    click util7 "https://github.com/facebookresearch/spinquant/blob/main/utils/fuse_norm_utils.py"
    click util8 "https://github.com/facebookresearch/spinquant/blob/main/utils/utils.py"

    %% Click Events for Script Wrappers and Execution Pipelines
    click scr1 "https://github.com/facebookresearch/spinquant/blob/main/scripts/10_optimize_rotation.sh"
    click scr2 "https://github.com/facebookresearch/spinquant/blob/main/scripts/11_optimize_rotation_fsdp.sh"
    click scr3 "https://github.com/facebookresearch/spinquant/blob/main/scripts/2_eval_ptq.sh"
    click scr4 "https://github.com/facebookresearch/spinquant/blob/main/train_utils/main.py"
    click scr5 "https://github.com/facebookresearch/spinquant/blob/main/eval_utils/main.py"
```
