# deadat0m/lsqfakequantize-pytorch architecture

> Architecture diagram of the GitHub repository deadat0m/lsqfakequantize-pytorch, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-06-06). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/deadat0m/lsqfakequantize-pytorch
- Repository: https://github.com/deadat0m/lsqfakequantize-pytorch

## Overview

##### Project type and purpose
- This is a PyTorch extension library (“torchlsq”) that implements Learned Step Size Quantization (LSQ / LSQ+) as a drop-in replacement for PyTorch’s FakeQuantize observers.
- It consists of a Python API layer, a build pipeline (setup.py & torch_patch.py), and a C++/CUDA core implementing custom autograd operators and kernels.

##### File structure
Top-level:
- .gitignore, README.md – documentation and ignore rules
- setup.py, setup_utils.py – setuptools build scripts
- torch_patch.py – a patcher for PyTorch headers
Python package “torchlsq”:
- __init__.py – package entry point
- extension.py – dynamic build & load of C++ extension via torch.utils.cpp_extension
- functional.py – Python-level functional API (lsq_forward, lsq_backward helpers)
- quantized/… – custom nn.Modules (LSQFakeQuantizer) that integrate with torch.quantization
Native code under torchlsq/csrc:
- macros.h, global_scope.h – shared definitions
- ops/autograd/lsq_autograd.cpp – custom AutogradFunction entry points
- ops/cpu/lsq_cpu.cpp – CPU reference implementation
- ops/cuda/lsq_cuda.cu – CUDA kernels
- ops/kernels/lsq_kernel.h – generic kernel declarations
- ops/lsq.cpp / lsq.h – dispatcher routing between CPU/CUDA
- torchlsq.cpp / torchlsq.h – PyBind or torch extension binding definitions

##### Key README insights
- Purpose: implement LSQ/LSQ+ quantization observers compatible with torch.quantization.
- Build requirements: Python ≥3.x, PyTorch ≥1.7, compiler with C++17, optional CUDA.
- Installation flow: patch PyTorch header → compile C++/CUDA extension → install package.
- Runtime: users configure QConfig with LSQFakeQuantizer modules, then run forward/backward to learn scale/shift.

##### Main components
a. Build & Installation Layer
- torch_patch.py → patches PyTorch C++ headers
- setup.py (with setup_utils.py) → compiles and links C++/CUDA code into a shared library (torchlsq extension)
b. Python API Layer
- torchlsq.extension – invokes torch.utils.cpp_extension.load to build & load the native module at import time
- torchlsq.functional – pure-Python helpers for forward/backward calls
- torchlsq.quantized.modules – LSQFakeQuantizer nn.Modules integrating with torch.quantization QConfig
c. Native Extension Layer
- torchlsq C++ extension entry points (torchlsq.cpp)
- Autograd functions (lsq_autograd.cpp) registered as torch::autograd::Function
- Dispatcher (lsq.cpp) choosing CPU vs. CUDA paths
- CPU kernels (lsq_cpu.cpp)
- CUDA kernels (lsq_cuda.cu)
d. External dependencies
- Host PyTorch library (ATen/TorchScript)
- CUDA runtime & driver (if GPU is used)

##### Relationships & data flow
- During installation, setup.py calls torch_patch.py → modifies PyTorch headers → compiles csrc into a shared object (.so).
- At runtime, Python import of torchlsq.extension → dynamic load of the .so into the Python process.
- User code sets up torch.quantization.QConfig with LSQFakeQuantizer modules.
- On forward:
Python LSQFakeQuantizer.forward → torchlsq.functional.quantize_dequantize → call into C++ autograd binding → dispatch to CPU or CUDA kernel → returns dequantized tensor.
- On backward:
Autograd engine invokes C++ gradient code → returns gradients w.r.t. input, scale, shift.
- All scale/shift parameters live in Python nn.Module wrappers, are learnable via standard optimizer.

##### Architectural patterns & principles
- Layered modularity: clear separation between Python interface, build scripts, and native implementation.
- Plugin/extension pattern: integrates into PyTorch via the quantization observer API.
- Dispatcher pattern: single entry point routes calls to CPU vs. CUDA.
- Autograd extension: custom torch::autograd::Function for forward/backward coupling.

##### Final advice
- Be detailed: capture each module and its connections.
- Review the file tree to ensure every major folder/file group is represented as a component or sub-component.

## Diagram (Mermaid)

```mermaid
flowchart TD
    %% Build & Installation Layer
    subgraph "Build & Installation Layer"
        direction TB
        Setup_py["setup.py"]:::python
        Setup_utils["setup_utils.py"]:::python
        Torch_patch["torch_patch.py"]:::python
    end

    %% Native Extension
    subgraph "Native Extension Layer"
        direction TB
        subgraph "Bindings"
            Torchlsq_cpp["torchlsq.cpp"]:::cpp
            Torchlsq_h["torchlsq.h"]:::cpp
        end
        subgraph "Autograd Functions"
            Lsq_autograd["lsq_autograd.cpp"]:::cpp
        end
        subgraph "Dispatcher"
            Lsq_dispatch_cpp["lsq.cpp"]:::cpp
            Lsq_dispatch_h["lsq.h"]:::cpp
        end
        subgraph "Kernels"
            subgraph "Kernel Declarations"
                Lsq_kernel_h["lsq_kernel.h"]:::cpp
            end
            subgraph "CPU Implementation"
                Lsq_cpu_cpp["lsq_cpu.cpp"]:::cpp
            end
            subgraph "CUDA Implementation"
                Lsq_cuda_cu["lsq_cuda.cu"]:::cpp
            end
        end
        subgraph "Shared Definitions"
            Macros_h["macros.h"]:::cpp
            Global_scope_h["global_scope.h"]:::cpp
            Ops_h["ops.h"]:::cpp
        end
    end

    %% Python API Layer
    subgraph "Python API Layer"
        direction TB
        Extension_py["extension.py"]:::python
        Functional_py["functional.py"]:::python
        subgraph "Quantized Modules"
            Observers_py["observers.py"]:::python
        end
    end

    %% External Dependencies
    PyTorch["PyTorch Core"]:::external
    CUDA_RT["CUDA Runtime"]:::external
    Autograd_engine["PyTorch Autograd Engine"]:::external

    %% Build-time relationships
    Setup_py -->|patch headers| Torch_patch
    Torch_patch -.->|generate patched headers| Torchlsq_cpp
    Setup_py -->|compile & link| Torchlsq_cpp
    Setup_utils -->|helper routines| Setup_py

    %% Runtime load
    Extension_py -->|"load extension"| Torchlsq_cpp
    Functional_py -->|"quantize_dequantize() call"| Lsq_autograd
    Observers_py -->|"forward() call"| Functional_py

    %% Dispatch and kernels
    Lsq_autograd -->|"invoke autograd hooks"| Autograd_engine
    Autograd_engine -->|"call forward/backward"| Lsq_autograd
    Lsq_autograd -->|"dispatch()"| Lsq_dispatch_cpp
    Lsq_dispatch_cpp -->|"CPU path"| Lsq_cpu_cpp
    Lsq_dispatch_cpp -->|"CUDA path"| Lsq_cuda_cu
    CUDA_RT -->|"kernel launch"| Lsq_cuda_cu

    %% Integration with shared definitions
    Lsq_dispatch_cpp --> Globalsub
    Lsq_cpu_cpp --> Globalsub
    Lsq_cuda_cu --> Globalsub
    Lsq_autograd --> Globalsub

    subgraph Globalsub["Shared Definitions"]
        direction TB
        Macros_h
        Global_scope_h
        Ops_h
    end

    %% Click events
    click Setup_py "https://github.com/deadat0m/lsqfakequantize-pytorch/blob/master/setup.py"
    click Setup_utils "https://github.com/deadat0m/lsqfakequantize-pytorch/blob/master/setup_utils.py"
    click Torch_patch "https://github.com/deadat0m/lsqfakequantize-pytorch/blob/master/torch_patch.py"
    click Extension_py "https://github.com/deadat0m/lsqfakequantize-pytorch/blob/master/torchlsq/extension.py"
    click Functional_py "https://github.com/deadat0m/lsqfakequantize-pytorch/blob/master/torchlsq/functional.py"
    click Observers_py "https://github.com/deadat0m/lsqfakequantize-pytorch/blob/master/torchlsq/quantized/modules/observers.py"
    click Torchlsq_cpp "https://github.com/deadat0m/lsqfakequantize-pytorch/blob/master/torchlsq/csrc/torchlsq.cpp"
    click Torchlsq_h "https://github.com/deadat0m/lsqfakequantize-pytorch/blob/master/torchlsq/csrc/torchlsq.h"
    click Lsq_autograd "https://github.com/deadat0m/lsqfakequantize-pytorch/blob/master/torchlsq/csrc/ops/autograd/lsq_autograd.cpp"
    click Lsq_dispatch_cpp "https://github.com/deadat0m/lsqfakequantize-pytorch/blob/master/torchlsq/csrc/ops/lsq.cpp"
    click Lsq_dispatch_h "https://github.com/deadat0m/lsqfakequantize-pytorch/blob/master/torchlsq/csrc/ops/lsq.h"
    click Lsq_cpu_cpp "https://github.com/deadat0m/lsqfakequantize-pytorch/blob/master/torchlsq/csrc/ops/cpu/lsq_cpu.cpp"
    click Lsq_cuda_cu "https://github.com/deadat0m/lsqfakequantize-pytorch/blob/master/torchlsq/csrc/ops/cuda/lsq_cuda.cu"
    click Lsq_kernel_h "https://github.com/deadat0m/lsqfakequantize-pytorch/blob/master/torchlsq/csrc/ops/kernels/lsq_kernel.h"
    click Macros_h "https://github.com/deadat0m/lsqfakequantize-pytorch/blob/master/torchlsq/csrc/macros.h"
    click Global_scope_h "https://github.com/deadat0m/lsqfakequantize-pytorch/blob/master/torchlsq/csrc/ops/global_scope.h"
    click Ops_h "https://github.com/deadat0m/lsqfakequantize-pytorch/blob/master/torchlsq/csrc/ops/ops.h"

    %% Styles
    classDef python fill:#bbdefb,stroke:#333,stroke-width:1px
    classDef cpp fill:#c8e6c9,stroke:#333,stroke-width:1px
    classDef external fill:#f5f5f5,stroke:#666,stroke-width:1px,stroke-dasharray: 2 2
```
