# tencent/ncnn architecture

> Architecture diagram of the GitHub repository tencent/ncnn, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-10-07). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/tencent/ncnn
- Repository: https://github.com/tencent/ncnn

## Overview

ncnn is a cross-platform neural-network inference framework, with CPU and Vulkan execution and a Python binding. The principal workflow is to convert a trained model (the README recommends PyTorch through pnnx), load its parameter and weight files into ncnn, provide input tensors, and extract output tensors. The repository also includes other format converters, optimization and quantization tools, and Python model-zoo helpers. Runtime internals are summarized by their tensor, layer, device, and pipeline responsibilities. Some tool and model-zoo links are documented only at repository level; those relationships are shown as unverified where source evidence is absent.

Main entry points: [`src/net.h`](https://github.com/tencent/ncnn/blob/master/src/net.h) (Network / extractor), [`tools/quantize/ncnnllm2table.cpp`](https://github.com/tencent/ncnn/blob/master/tools/quantize/ncnnllm2table.cpp) (LLM quantization), [`tools/pnnx/src/main.cpp`](https://github.com/tencent/ncnn/blob/master/tools/pnnx/src/main.cpp) (pnnx converter), [`src/layer.h`](https://github.com/tencent/ncnn/blob/master/src/layer.h) (Layer interface)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_conversion["Model conversion"]
  node_pnnx["pnnx converter<br/>[main.cpp]"]
  node_pnnx_ir["Graph IR<br/>[ir.h]"]
  node_pnnx_passes["Graph passes<br/>[pass_level2.h]"]
  node_ncnn_writer["ncnn serializer<br/>[save_ncnn.h]"]
  node_format_converters["Format converters<br/>[onnx2ncnn.cpp]"]
end

subgraph group_runtime["Inference runtime"]
  node_net["Network / extractor<br/>[net.h]"]
  node_tensor["Tensor storage<br/>[mat.h]"]
  node_layer["Layer interface<br/>[layer.h]"]
  node_layer_types["Layer registry types<br/>[layer_type.h]"]
  node_allocator["Memory allocator<br/>[allocator.h]"]
  node_data_reader["Model data reader<br/>[datareader.h]"]
  node_options["Runtime options<br/>[option.h]"]
  node_gpu["Vulkan device<br/>[gpu.h]"]
  node_pipeline["Vulkan pipeline<br/>[pipeline.h]"]
  node_pipeline_cache[("Pipeline cache<br/>[pipelinecache.h]")]
end

subgraph group_tools["Model tools"]
  node_model_optimizer["Model optimizer<br/>[ncnnoptimize.cpp]"]
  node_quant_tools["LLM quantization<br/>[ncnnllm2table.cpp]"]
  node_quant_apply["Quantization apply<br/>[ncnnllm2int.cpp]"]
  node_model_writer["Model writer<br/>[modelwriter.h]"]
  node_quant_lib["LLM quant helpers<br/>[ncnnllm_quant.h]"]
  node_quant_array["NumPy file support<br/>[npy.hpp]"]
end

subgraph group_python["Python interface"]
  node_python_binding["Python binding<br/>[main.cpp]"]
  node_model_zoo["Python model zoo<br/>[model_zoo.py]"]
  node_model_store["Model store<br/>[model_store.py]"]
  node_python_helpers["Python utilities<br/>[functional.py]"]
end

node_developer(("Model developer"))
node_pytorch["PyTorch / ONNX"]
node_model_files["ncnn model files"]
node_application(("Deploying application"))

node_developer -->|"exports model"| node_pytorch
node_pytorch -->|"converts through"| node_pnnx
node_pnnx -->|"loads graph"| node_pnnx_ir
node_pnnx -->|"runs passes"| node_pnnx_passes
node_pnnx -->|"saves model"| node_ncnn_writer
node_ncnn_writer -.->|"writes files"| node_model_files
node_pytorch -.->|"imports models"| node_format_converters
node_format_converters -.->|"writes model"| node_model_files
node_model_files -.->|"optimizes model"| node_model_optimizer
node_model_optimizer -.->|"rewrites model"| node_model_files
node_model_files -.->|"analyzes model"| node_quant_tools
node_quant_tools -->|"uses writer"| node_model_writer
node_quant_tools -->|"uses helpers"| node_quant_lib
node_quant_tools -->|"uses arrays"| node_quant_array
node_quant_tools -.->|"provides table"| node_quant_apply
node_quant_apply -->|"uses writer"| node_model_writer
node_model_files -->|"loads parameters"| node_net
node_application -->|"runs inference"| node_net
node_net -->|"accepts tensors"| node_tensor
node_net -->|"uses layers"| node_layer
node_net -->|"reads model data"| node_data_reader
node_net -->|"configures execution"| node_options
node_tensor -->|"allocates storage"| node_allocator
node_layer -->|"processes tensors"| node_tensor
node_layer -.->|"registers types"| node_layer_types
node_application -.->|"selects backend"| node_gpu
node_gpu -->|"creates pipelines"| node_pipeline
node_pipeline -.->|"uses cache"| node_pipeline_cache
node_application -->|"calls Python API"| node_python_binding
node_python_binding -.->|"exposes inference"| node_net
node_developer -.->|"uses pretrained models"| node_model_zoo
node_model_zoo -->|"looks up models"| node_model_store
node_model_zoo -->|"uses helpers"| node_python_helpers

click node_pnnx "https://github.com/tencent/ncnn/blob/master/tools/pnnx/src/main.cpp"
click node_pnnx_ir "https://github.com/tencent/ncnn/blob/master/tools/pnnx/src/ir.h"
click node_pnnx_passes "https://github.com/tencent/ncnn/blob/master/tools/pnnx/src/pass_level2.h"
click node_ncnn_writer "https://github.com/tencent/ncnn/blob/master/tools/pnnx/src/save_ncnn.h"
click node_format_converters "https://github.com/tencent/ncnn/blob/master/tools/onnx/onnx2ncnn.cpp"
click node_model_optimizer "https://github.com/tencent/ncnn/blob/master/tools/ncnnoptimize.cpp"
click node_quant_tools "https://github.com/tencent/ncnn/blob/master/tools/quantize/ncnnllm2table.cpp"
click node_quant_apply "https://github.com/tencent/ncnn/blob/master/tools/quantize/ncnnllm2int.cpp"
click node_model_writer "https://github.com/tencent/ncnn/blob/master/tools/modelwriter.h"
click node_python_binding "https://github.com/tencent/ncnn/blob/master/python/src/main.cpp"
click node_model_zoo "https://github.com/tencent/ncnn/blob/master/python/ncnn/model_zoo/model_zoo.py"
click node_model_store "https://github.com/tencent/ncnn/blob/master/python/ncnn/model_zoo/model_store.py"
click node_python_helpers "https://github.com/tencent/ncnn/blob/master/python/ncnn/utils/functional.py"
click node_net "https://github.com/tencent/ncnn/blob/master/src/net.h"
click node_tensor "https://github.com/tencent/ncnn/blob/master/src/mat.h"
click node_layer "https://github.com/tencent/ncnn/blob/master/src/layer.h"
click node_layer_types "https://github.com/tencent/ncnn/blob/master/src/layer_type.h"
click node_allocator "https://github.com/tencent/ncnn/blob/master/src/allocator.h"
click node_data_reader "https://github.com/tencent/ncnn/blob/master/src/datareader.h"
click node_options "https://github.com/tencent/ncnn/blob/master/src/option.h"
click node_gpu "https://github.com/tencent/ncnn/blob/master/src/gpu.h"
click node_pipeline "https://github.com/tencent/ncnn/blob/master/src/pipeline.h"
click node_pipeline_cache "https://github.com/tencent/ncnn/blob/master/src/pipelinecache.h"
click node_quant_lib "https://github.com/tencent/ncnn/blob/master/tools/quantize/ncnnllm_quant.h"
click node_quant_array "https://github.com/tencent/ncnn/blob/master/tools/quantize/npy.hpp"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_pnnx,node_pnnx_ir,node_pnnx_passes,node_ncnn_writer,node_format_converters toneBlue
class node_net,node_tensor,node_layer,node_layer_types,node_allocator,node_data_reader,node_options,node_gpu,node_pipeline,node_pipeline_cache toneAmber
class node_model_optimizer,node_quant_tools,node_quant_apply,node_model_writer,node_quant_lib,node_quant_array toneMint
class node_python_binding,node_model_zoo,node_model_store,node_python_helpers toneRose
class node_developer,node_pytorch,node_model_files,node_application toneIndigo
```

## Components

### Model conversion

- [pnnx converter](https://github.com/tencent/ncnn/blob/master/tools/pnnx/src/main.cpp): `tools/pnnx/src/main.cpp`
- [Graph IR](https://github.com/tencent/ncnn/blob/master/tools/pnnx/src/ir.h): `tools/pnnx/src/ir.h`
- [Graph passes](https://github.com/tencent/ncnn/blob/master/tools/pnnx/src/pass_level2.h): `tools/pnnx/src/pass_level2.h`
- [ncnn serializer](https://github.com/tencent/ncnn/blob/master/tools/pnnx/src/save_ncnn.h): `tools/pnnx/src/save_ncnn.h`
- [Format converters](https://github.com/tencent/ncnn/blob/master/tools/onnx/onnx2ncnn.cpp): `tools/onnx/onnx2ncnn.cpp`

### Inference runtime

- [Network / extractor](https://github.com/tencent/ncnn/blob/master/src/net.h): `src/net.h`
- [Tensor storage](https://github.com/tencent/ncnn/blob/master/src/mat.h): `src/mat.h`
- [Layer interface](https://github.com/tencent/ncnn/blob/master/src/layer.h): `src/layer.h`
- [Layer registry types](https://github.com/tencent/ncnn/blob/master/src/layer_type.h): `src/layer_type.h`
- [Memory allocator](https://github.com/tencent/ncnn/blob/master/src/allocator.h): `src/allocator.h`
- [Model data reader](https://github.com/tencent/ncnn/blob/master/src/datareader.h): `src/datareader.h`
- [Runtime options](https://github.com/tencent/ncnn/blob/master/src/option.h): `src/option.h`
- [Vulkan device](https://github.com/tencent/ncnn/blob/master/src/gpu.h): `src/gpu.h`
- [Vulkan pipeline](https://github.com/tencent/ncnn/blob/master/src/pipeline.h): `src/pipeline.h`
- [Pipeline cache](https://github.com/tencent/ncnn/blob/master/src/pipelinecache.h): `src/pipelinecache.h`

### Model tools

- [Model optimizer](https://github.com/tencent/ncnn/blob/master/tools/ncnnoptimize.cpp): `tools/ncnnoptimize.cpp`
- [LLM quantization](https://github.com/tencent/ncnn/blob/master/tools/quantize/ncnnllm2table.cpp): `tools/quantize/ncnnllm2table.cpp`
- [Quantization apply](https://github.com/tencent/ncnn/blob/master/tools/quantize/ncnnllm2int.cpp): `tools/quantize/ncnnllm2int.cpp`
- [Model writer](https://github.com/tencent/ncnn/blob/master/tools/modelwriter.h): `tools/modelwriter.h`
- [LLM quant helpers](https://github.com/tencent/ncnn/blob/master/tools/quantize/ncnnllm_quant.h): `tools/quantize/ncnnllm_quant.h`
- [NumPy file support](https://github.com/tencent/ncnn/blob/master/tools/quantize/npy.hpp): `tools/quantize/npy.hpp`

### Python interface

- [Python binding](https://github.com/tencent/ncnn/blob/master/python/src/main.cpp): `python/src/main.cpp`
- [Python model zoo](https://github.com/tencent/ncnn/blob/master/python/ncnn/model_zoo/model_zoo.py): `python/ncnn/model_zoo/model_zoo.py`
- [Model store](https://github.com/tencent/ncnn/blob/master/python/ncnn/model_zoo/model_store.py): `python/ncnn/model_zoo/model_store.py`
- [Python utilities](https://github.com/tencent/ncnn/blob/master/python/ncnn/utils/functional.py): `python/ncnn/utils/functional.py`

### Other components

- **Model developer**
- **PyTorch / ONNX**
- **ncnn model files**
- **Deploying application**

## Connections

- Model developer → PyTorch / ONNX: exports model (evidence: `README.md`)
- PyTorch / ONNX → pnnx converter: converts through (evidence: `README.md`)
- pnnx converter → Graph IR: loads graph (evidence: `tools/pnnx/src/main.cpp`)
- pnnx converter → Graph passes: runs passes (evidence: `tools/pnnx/src/main.cpp`)
- pnnx converter → ncnn serializer: saves model (evidence: `tools/pnnx/src/main.cpp`)
- ncnn serializer → ncnn model files: writes files
- PyTorch / ONNX → Format converters: imports models
- Format converters → ncnn model files: writes model
- ncnn model files → Model optimizer: optimizes model
- Model optimizer → ncnn model files: rewrites model
- ncnn model files → LLM quantization: analyzes model
- LLM quantization → Model writer: uses writer (evidence: `tools/quantize/ncnnllm2table.cpp`)
- LLM quantization → LLM quant helpers: uses helpers (evidence: `tools/quantize/ncnnllm2table.cpp`)
- LLM quantization → NumPy file support: uses arrays (evidence: `tools/quantize/ncnnllm2table.cpp`)
- LLM quantization → Quantization apply: provides table
- Quantization apply → Model writer: uses writer (evidence: `tools/quantize/ncnnllm2int.cpp`)
- ncnn model files → Network / extractor: loads parameters (evidence: `README.md`)
- Deploying application → Network / extractor: runs inference (evidence: `README.md`)
- Network / extractor → Tensor storage: accepts tensors (evidence: `README.md`)
- Network / extractor → Layer interface: uses layers (evidence: `src/net.h`)
- Network / extractor → Model data reader: reads model data (evidence: `src/net.h`)
- Network / extractor → Runtime options: configures execution (evidence: `src/net.h`)
- Tensor storage → Memory allocator: allocates storage (evidence: `src/mat.h`)
- Layer interface → Tensor storage: processes tensors (evidence: `src/layer.h`)
- Layer interface → Layer registry types: registers types
- Deploying application → Vulkan device: selects backend
- Vulkan device → Vulkan pipeline: creates pipelines (evidence: `src/pipeline.h`)
- Vulkan pipeline → Pipeline cache: uses cache
- Deploying application → Python binding: calls Python API (evidence: `README.md`)
- Python binding → Network / extractor: exposes inference
- Model developer → Python model zoo: uses pretrained models
- Python model zoo → Model store: looks up models (evidence: `python/ncnn/model_zoo/model_store.py`)
- Python model zoo → Python utilities: uses helpers
