How tencent/ncnn works
ncnn is a cross-platform neural-network inference framework, with CPU and Vulkan execution and a Python binding. The principal workflow is to convert a trained model (the README recommends PyTorch through pnnx), load its parameter and weight files into ncnn, provide input tensors, and extract output tensors. The repository also includes other format converters, optimization and quantization tools, and Python model-zoo helpers. Runtime internals are summarized by their tensor, layer, device, and pipeline responsibilities. Some tool and model-zoo links are documented only at repository level; those relationships are shown as unverified where source evidence is absent.
Main entry points: net.h Network / extractor · ncnnllm2table.cpp LLM quantization · main.cpp pnnx converter · layer.h Layer interface
Components 29
Model conversion
- pnnx converter
tools/pnnx/src/main.cpp - Graph IR
tools/pnnx/src/ir.h - Graph passes
tools/pnnx/src/pass_level2.h - ncnn serializer
tools/pnnx/src/save_ncnn.h - Format converters
tools/onnx/onnx2ncnn.cpp
Inference runtime
- Network / extractor
src/net.h - Tensor storage
src/mat.h - Layer interface
src/layer.h - Layer registry types
src/layer_type.h - Memory allocator
src/allocator.h - Model data reader
src/datareader.h - Runtime options
src/option.h - Vulkan device
src/gpu.h - Vulkan pipeline
src/pipeline.h - Pipeline cache
src/pipelinecache.h
Model tools
- Model optimizer
tools/ncnnoptimize.cpp - LLM quantization
tools/quantize/ncnnllm2table.cpp - Quantization apply
tools/quantize/ncnnllm2int.cpp - Model writer
tools/modelwriter.h - LLM quant helpers
tools/quantize/ncnnllm_quant.h - NumPy file support
tools/quantize/npy.hpp
Python interface
- Python binding
python/src/main.cpp - Python model zoo
python/ncnn/model_zoo/model_zoo.py - Model store
python/ncnn/model_zoo/model_store.py - Python utilities
python/ncnn/utils/functional.py
Other components
- Model developer
- PyTorch / ONNX
- ncnn model files
- Deploying application
Connections 33
- Model developer → PyTorch / ONNX: exports model
README.md - PyTorch / ONNX → pnnx converter: converts through
README.md - pnnx converter → Graph IR: loads graph
tools/pnnx/src/main.cpp - pnnx converter → Graph passes: runs passes
tools/pnnx/src/main.cpp - pnnx converter → ncnn serializer: saves model
tools/pnnx/src/main.cpp - ncnn serializer → ncnn model files: writes files
- PyTorch / ONNX → Format converters: imports models
- Format converters → ncnn model files: writes model
- ncnn model files → Model optimizer: optimizes model
- Model optimizer → ncnn model files: rewrites model
- ncnn model files → LLM quantization: analyzes model
- LLM quantization → Model writer: uses writer
tools/quantize/ncnnllm2table.cpp - LLM quantization → LLM quant helpers: uses helpers
tools/quantize/ncnnllm2table.cpp - LLM quantization → NumPy file support: uses arrays
tools/quantize/ncnnllm2table.cpp - LLM quantization → Quantization apply: provides table
- Quantization apply → Model writer: uses writer
tools/quantize/ncnnllm2int.cpp - ncnn model files → Network / extractor: loads parameters
README.md - Deploying application → Network / extractor: runs inference
README.md - Network / extractor → Tensor storage: accepts tensors
README.md - Network / extractor → Layer interface: uses layers
src/net.h - Network / extractor → Model data reader: reads model data
src/net.h - Network / extractor → Runtime options: configures execution
src/net.h - Tensor storage → Memory allocator: allocates storage
src/mat.h - Layer interface → Tensor storage: processes tensors
src/layer.h - Layer interface → Layer registry types: registers types
- Deploying application → Vulkan device: selects backend
- Vulkan device → Vulkan pipeline: creates pipelines
src/pipeline.h - Vulkan pipeline → Pipeline cache: uses cache
- Deploying application → Python binding: calls Python API
README.md - Python binding → Network / extractor: exposes inference
- Model developer → Python model zoo: uses pretrained models
- Python model zoo → Model store: looks up models
python/ncnn/model_zoo/model_store.py - Python model zoo → Python utilities: uses helpers