How openvinotoolkit/nncf works
NNCF is a neural-network compression framework for PyTorch, TorchFX, ONNX, and OpenVINO models. Its principal documented workflow accepts a model and calibration dataset through the quantization API, collects model statistics, applies a backend-aware compression pipeline, and returns a compressed model. Other major branches cover weight compression, training-time quantization and pruning, and experimental activation sparsity. The graph follows the README and supplied source excerpts; details of unsampled implementations and their wiring are represented only where documented.
Main entry points: pipeline.py Pipeline executor · nncf Compression API · model_transformer.py Graph transformation · engine.py Inference engines
Components 26
User API
- Compression API
src/nncf - Dataset wrapper
src/nncf/data/dataset.py
Shared compression core
- PTQ pipeline
src/nncf/quantization/algorithms/post_training/pipeline.py - Pipeline executor
src/nncf/quantization/algorithms/pipeline.py - Model graph
src/nncf/common/graph/graph.py - Statistics collection
src/nncf/common/tensor_statistics/aggregator.py - Graph transformation
src/nncf/common/graph/model_transformer.py
Compression algorithms
- Min-max quantization
src/nncf/quantization/algorithms/min_max/algorithm.py - SmoothQuant
src/nncf/quantization/algorithms/smooth_quant/algorithm.py - Bias correction
src/nncf/quantization/algorithms/fast_bias_correction/algorithm.py - Weight compression
src/nncf/quantization/algorithms/weight_compression/algorithm.py - Accuracy control
src/nncf/quantization/algorithms/accuracy_control/algorithm.py
Model backends
- PyTorch support
src/nncf/torch - TorchFX support
src/nncf/experimental/torch/fx - ONNX support
src/nncf/onnx - OpenVINO support
src/nncf/openvino - Inference engines
src/nncf/common/engine.py - Model export
src/nncf/torch
Training and extras
- Quantization-aware training
src/nncf/torch/function_hook/quantization/quantize_model.py - Pruning
src/nncf/torch/function_hook/pruning/prune_model.py - Activation sparsity
src/nncf/experimental/torch/sparsify_activations/sparsify_activations_impl.py
Other components
- Model user
- Input model
- Calibration data
- Backend runtimes
- Compressed model
Connections 31
- Model user → Compression API: configures
- Model user → Input model: provides
- Model user → Calibration data: provides
- Compression API → Dataset wrapper: wraps data
- Compression API → PTQ pipeline: starts PTQ
- PTQ pipeline → Pipeline executor: creates steps
- Pipeline executor → Model graph: builds graph
- Pipeline executor → Statistics collection: collects stats
- Dataset wrapper → Statistics collection: supplies samples
- Pipeline executor → SmoothQuant: runs step
- Pipeline executor → Min-max quantization: runs step
- Pipeline executor → Bias correction: runs step
- Pipeline executor → Weight compression: runs compression
- Pipeline executor → Accuracy control: optional control
- Min-max quantization → Graph transformation: transforms model
- SmoothQuant → Graph transformation: transforms model
- Bias correction → Graph transformation: corrects model
- Compression API → PyTorch support: dispatches backend
- Compression API → TorchFX support: dispatches backend
- Compression API → ONNX support: dispatches backend
- Compression API → OpenVINO support: dispatches backend
- PyTorch support → Inference engines: provides inference
- ONNX support → Inference engines: provides inference
- OpenVINO support → Inference engines: provides inference
- Inference engines → Backend runtimes: runs inference
- Graph transformation → Compressed model: returns model
- Model user → Quantization-aware training: fine-tunes
- Model user → Pruning: configures
- Model user → Activation sparsity: opts into
- PyTorch support → Model export: exports model
- Model export → ONNX support: exports to