How xilinx/brevitas works
Project type and purpose
• Brevitas is an open-source Python library (built on PyTorch) for neural network quantization.
• It provides both PTQ (post-training quantization) and QAT (quantization-aware training) workflows, plus export to ONNX/QONNX and example pipelines.
Main high-level components
a. Core Quantization API
– brevitas.core (bit-width, function wrappers, quant, scaling, stats, utils)
– brevitas.quant (quantization algorithms, solvers)
– brevitas.quant_tensor (QuantTensor classes & handlers)
b. PyTorch Integration
– brevitas.nn (Quantized layer modules)
– brevitas.fx (FX tracer & value tracer)
– brevitas.proxy (parameter/runtime proxies)
c. Graph & Compiler
– brevitas.graph (graph transforms: quantize, equalize, calibrate, etc.)
– brevitas.compiler (overall orchestration)
d. Export Subsystem
– brevitas.export.*
• ONNX / QONNX export managers & handlers
• Torch export (QCDQ)
• Inference export
e. Examples & User Flows
– brevitas_examples (ImageNet PTQ, LLM quant, speech, diffusion, etc.)
f. External Dependencies
– PyTorch, ONNX, numpy, SciPy, third-party quant libs (QONNX), hardware backends (TVM, FINN)
g. CI / Docs
– .github/workflows, docs/, docsrc/
Component relationships & interactions
1. User code (examples or custom scripts) instantiates brevitas.nn.Quant* layers → these layers wrap native PyTorch modules.
2. Internally, Quant* layers use brevitas.proxy to wrap parameters/activations as QuantTensor.
3. On forward pass, QuantTensor interacts with brevitas.core.function_wrapper and quant routines in brevitas.core.quant.
4. For FX-based workflows, brevitas.fx tracer inspects the layer graph → feeds into brevitas.graph transforms (quantize, equalize, calibrate).
5. After calibration/QAT, brevitas.export uses handlers/managers to produce ONNX/QONNX or Torch+QCDQ modules.
6. Exported models run in inference engines or hardware accelerators (e.g., TVM, FINN).
7. Examples drive and demonstrate these flows.
Architectural patterns & principles
• Modular layering: core logic ←→ integration layer ←→ export
• Dependency injection & handler registries (common/handler) for extensibility
• FX-based graph rewrites for quant transforms
• Proxy/Tensor wrapper pattern separating quant logic from PyTorch internals
Detailed breaking-down of components (sample decomposition)
[User Scripts / Examples] → [Integration Layer (nn, proxy, quant_tensor)] → [Core Quant API] ↔ [FX & Graph Passes] → [Export Subsystem] → [External Runtime / Hardware]