How deadat0m/lsqfakequantize-pytorch works
Project type and purpose
• This is a PyTorch extension library (“torchlsq”) that implements Learned Step Size Quantization (LSQ / LSQ+) as a drop-in replacement for PyTorch’s FakeQuantize observers.
• It consists of a Python API layer, a build pipeline (setup.py & torch_patch.py), and a C++/CUDA core implementing custom autograd operators and kernels.
File structure
Top-level:
• .gitignore, README.md – documentation and ignore rules
• setup.py, setup_utils.py – setuptools build scripts
• torch_patch.py – a patcher for PyTorch headers
Python package “torchlsq”:
• __init__.py – package entry point
• extension.py – dynamic build & load of C++ extension via torch.utils.cpp_extension
• functional.py – Python-level functional API (lsq_forward, lsq_backward helpers)
• quantized/… – custom nn.Modules (LSQFakeQuantizer) that integrate with torch.quantization
Native code under torchlsq/csrc:
• macros.h, global_scope.h – shared definitions
• ops/autograd/lsq_autograd.cpp – custom AutogradFunction entry points
• ops/cpu/lsq_cpu.cpp – CPU reference implementation
• ops/cuda/lsq_cuda.cu – CUDA kernels
• ops/kernels/lsq_kernel.h – generic kernel declarations
• ops/lsq.cpp / lsq.h – dispatcher routing between CPU/CUDA
• torchlsq.cpp / torchlsq.h – PyBind or torch extension binding definitions
Key README insights
• Purpose: implement LSQ/LSQ+ quantization observers compatible with torch.quantization.
• Build requirements: Python ≥3.x, PyTorch ≥1.7, compiler with C++17, optional CUDA.
• Installation flow: patch PyTorch header → compile C++/CUDA extension → install package.
• Runtime: users configure QConfig with LSQFakeQuantizer modules, then run forward/backward to learn scale/shift.
Main components
a. Build & Installation Layer
• torch_patch.py → patches PyTorch C++ headers
• setup.py (with setup_utils.py) → compiles and links C++/CUDA code into a shared library (torchlsq extension)
b. Python API Layer
• torchlsq.extension – invokes torch.utils.cpp_extension.load to build & load the native module at import time
• torchlsq.functional – pure-Python helpers for forward/backward calls
• torchlsq.quantized.modules – LSQFakeQuantizer nn.Modules integrating with torch.quantization QConfig
c. Native Extension Layer
• torchlsq C++ extension entry points (torchlsq.cpp)
• Autograd functions (lsq_autograd.cpp) registered as torch::autograd::Function
• Dispatcher (lsq.cpp) choosing CPU vs. CUDA paths
• CPU kernels (lsq_cpu.cpp)
• CUDA kernels (lsq_cuda.cu)
d. External dependencies
• Host PyTorch library (ATen/TorchScript)
• CUDA runtime & driver (if GPU is used)
Relationships & data flow
• During installation, setup.py calls torch_patch.py → modifies PyTorch headers → compiles csrc into a shared object (.so).
• At runtime, Python import of torchlsq.extension → dynamic load of the .so into the Python process.
• User code sets up torch.quantization.QConfig with LSQFakeQuantizer modules.
• On forward:
Python LSQFakeQuantizer.forward → torchlsq.functional.quantize_dequantize → call into C++ autograd binding → dispatch to CPU or CUDA kernel → returns dequantized tensor.
• On backward:
Autograd engine invokes C++ gradient code → returns gradients w.r.t. input, scale, shift.
• All scale/shift parameters live in Python nn.Module wrappers, are learnable via standard optimizer.
Architectural patterns & principles
• Layered modularity: clear separation between Python interface, build scripts, and native implementation.
• Plugin/extension pattern: integrates into PyTorch via the quantization observer API.
• Dispatcher pattern: single entry point routes calls to CPU vs. CUDA.
• Autograd extension: custom torch::autograd::Function for forward/backward coupling.
Final advice
• Be detailed: capture each module and its connections.
• Review the file tree to ensure every major folder/file group is represented as a component or sub-component.