# mit-han-lab/nunchaku architecture

> Architecture diagram of the GitHub repository mit-han-lab/nunchaku, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-10-01). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/mit-han-lab/nunchaku
- Repository: https://github.com/mit-han-lab/nunchaku

## Overview

Nunchaku is a GPU inference engine for quantized neural networks, particularly image-generation models. A typical user-facing workflow is to launch a model-specific demo, run inference through a Nunchaku transformer, and receive generated images; the repository also provides LoRA conversion, caching, and PuLID personalization. The native C++/CUDA engine and Python model layers implement the inference core. Edges are limited to relationships shown in excerpts, the source index, or README statements; several listed model and demo files were not sampled, so their internal wiring is not inferred.

Main entry points: [`app/flux.1/depth_canny/run_gradio.py`](https://github.com/mit-han-lab/nunchaku/blob/main/app/flux.1/depth_canny/run_gradio.py) (FLUX demos), [`src/FluxModel.cpp`](https://github.com/mit-han-lab/nunchaku/blob/main/src/FluxModel.cpp) (FLUX engine), [`nunchaku/models/transformers/transformer_flux.py`](https://github.com/mit-han-lab/nunchaku/blob/main/nunchaku/models/transformers/transformer_flux.py) (FLUX transformer), [`nunchaku/pipeline/pipeline_flux_pulid.py`](https://github.com/mit-han-lab/nunchaku/blob/main/nunchaku/pipeline/pipeline_flux_pulid.py) (PuLID pipeline)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_interfaces["Demos and workflows"]
  node_flux_demo["FLUX demos<br/>[run_gradio.py]"]
  node_sana_demo["Sana demo<br/>[run_gradio.py]"]
end

subgraph group_models["Model inference"]
  node_flux_model["FLUX transformer"]
  node_model_family["Other transformers"]
  node_linear["Quantized linear<br/>[linear.py]"]
end

subgraph group_features["Inference features"]
  node_lora_convert["LoRA conversion"]
  node_lora_pack["LoRA packing<br/>[packer.py]"]
  node_cache["Inference caching<br/>[fbcache.py]"]
  node_pulid_pipeline["PuLID pipeline"]
  node_pulid_encoder["Identity encoder"]
  node_safety["Safety checker<br/>[safety_checker.py]"]
end

subgraph group_engine["Native engine"]
  node_ops["Quantized ops<br/>[gemm.py]"]
  node_native_api["FLUX native API<br/>[flux.h]"]
  node_flux_engine["FLUX engine<br/>[FluxModel.cpp]"]
  node_tensor["Tensor runtime<br/>[Tensor.h]"]
  node_cuda_kernels["CUDA kernels<br/>[zgemm.h]"]
end

node_user(("User"))

node_user -->|"launches"| node_flux_demo
node_user -->|"launches"| node_sana_demo
node_flux_demo -->|"uses"| node_flux_model
node_flux_demo -->|"checks output"| node_safety
node_sana_demo -->|"checks output"| node_safety
node_flux_model -->|"uses conversion"| node_lora_convert
node_lora_convert -->|"uses packing"| node_lora_pack
node_linear -->|"calls"| node_ops
node_pulid_pipeline -->|"uses encoder"| node_pulid_encoder
node_pulid_pipeline -->|"uses transformer"| node_flux_model
node_native_api -->|"calls"| node_flux_engine
node_flux_engine -->|"uses kernels"| node_cuda_kernels
node_flux_engine -->|"uses tensors"| node_tensor

click node_flux_demo "https://github.com/mit-han-lab/nunchaku/blob/main/app/flux.1/depth_canny/run_gradio.py"
click node_sana_demo "https://github.com/mit-han-lab/nunchaku/blob/main/app/sana/t2i/run_gradio.py"
click node_flux_model "https://github.com/mit-han-lab/nunchaku/blob/main/nunchaku/models/transformers/transformer_flux.py"
click node_model_family "https://github.com/mit-han-lab/nunchaku/blob/main/nunchaku/models/transformers/transformer_qwenimage.py"
click node_linear "https://github.com/mit-han-lab/nunchaku/blob/main/nunchaku/models/linear.py"
click node_ops "https://github.com/mit-han-lab/nunchaku/blob/main/nunchaku/ops/gemm.py"
click node_lora_convert "https://github.com/mit-han-lab/nunchaku/blob/main/nunchaku/lora/flux/nunchaku_converter.py"
click node_lora_pack "https://github.com/mit-han-lab/nunchaku/blob/main/nunchaku/lora/flux/packer.py"
click node_cache "https://github.com/mit-han-lab/nunchaku/blob/main/nunchaku/caching/fbcache.py"
click node_pulid_pipeline "https://github.com/mit-han-lab/nunchaku/blob/main/nunchaku/pipeline/pipeline_flux_pulid.py"
click node_pulid_encoder "https://github.com/mit-han-lab/nunchaku/blob/main/nunchaku/models/pulid/encoders_transformer.py"
click node_safety "https://github.com/mit-han-lab/nunchaku/blob/main/nunchaku/models/safety_checker.py"
click node_native_api "https://github.com/mit-han-lab/nunchaku/blob/main/nunchaku/csrc/flux.h"
click node_flux_engine "https://github.com/mit-han-lab/nunchaku/blob/main/src/FluxModel.cpp"
click node_tensor "https://github.com/mit-han-lab/nunchaku/blob/main/src/Tensor.h"
click node_cuda_kernels "https://github.com/mit-han-lab/nunchaku/blob/main/src/kernels/zgemm/zgemm.h"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_flux_demo,node_sana_demo,node_user toneBlue
class node_flux_model,node_model_family,node_linear toneAmber
class node_lora_convert,node_lora_pack,node_cache,node_pulid_pipeline,node_pulid_encoder,node_safety toneMint
class node_ops,node_native_api,node_flux_engine,node_tensor,node_cuda_kernels toneRose
```

## Components

### Demos and workflows

- [FLUX demos](https://github.com/mit-han-lab/nunchaku/blob/main/app/flux.1/depth_canny/run_gradio.py): `app/flux.1/depth_canny/run_gradio.py`
- [Sana demo](https://github.com/mit-han-lab/nunchaku/blob/main/app/sana/t2i/run_gradio.py): `app/sana/t2i/run_gradio.py`

### Model inference

- [FLUX transformer](https://github.com/mit-han-lab/nunchaku/blob/main/nunchaku/models/transformers/transformer_flux.py): `nunchaku/models/transformers/transformer_flux.py`
- [Other transformers](https://github.com/mit-han-lab/nunchaku/blob/main/nunchaku/models/transformers/transformer_qwenimage.py): `nunchaku/models/transformers/transformer_qwenimage.py`
- [Quantized linear](https://github.com/mit-han-lab/nunchaku/blob/main/nunchaku/models/linear.py): `nunchaku/models/linear.py`

### Inference features

- [LoRA conversion](https://github.com/mit-han-lab/nunchaku/blob/main/nunchaku/lora/flux/nunchaku_converter.py): `nunchaku/lora/flux/nunchaku_converter.py`
- [LoRA packing](https://github.com/mit-han-lab/nunchaku/blob/main/nunchaku/lora/flux/packer.py): `nunchaku/lora/flux/packer.py`
- [Inference caching](https://github.com/mit-han-lab/nunchaku/blob/main/nunchaku/caching/fbcache.py): `nunchaku/caching/fbcache.py`
- [PuLID pipeline](https://github.com/mit-han-lab/nunchaku/blob/main/nunchaku/pipeline/pipeline_flux_pulid.py): `nunchaku/pipeline/pipeline_flux_pulid.py`
- [Identity encoder](https://github.com/mit-han-lab/nunchaku/blob/main/nunchaku/models/pulid/encoders_transformer.py): `nunchaku/models/pulid/encoders_transformer.py`
- [Safety checker](https://github.com/mit-han-lab/nunchaku/blob/main/nunchaku/models/safety_checker.py): `nunchaku/models/safety_checker.py`

### Native engine

- [Quantized ops](https://github.com/mit-han-lab/nunchaku/blob/main/nunchaku/ops/gemm.py): `nunchaku/ops/gemm.py`
- [FLUX native API](https://github.com/mit-han-lab/nunchaku/blob/main/nunchaku/csrc/flux.h): `nunchaku/csrc/flux.h`
- [FLUX engine](https://github.com/mit-han-lab/nunchaku/blob/main/src/FluxModel.cpp): `src/FluxModel.cpp`
- [Tensor runtime](https://github.com/mit-han-lab/nunchaku/blob/main/src/Tensor.h): `src/Tensor.h`
- [CUDA kernels](https://github.com/mit-han-lab/nunchaku/blob/main/src/kernels/zgemm/zgemm.h): `src/kernels/zgemm/zgemm.h`

### Other components

- **User**

## Connections

- User → FLUX demos: launches
- User → Sana demo: launches
- FLUX demos → FLUX transformer: uses (evidence: `app/flux.1/depth_canny/run_gradio.py`)
- FLUX demos → Safety checker: checks output (evidence: `app/flux.1/depth_canny/run_gradio.py`)
- Sana demo → Safety checker: checks output (evidence: `app/sana/t2i/run_gradio.py`)
- FLUX transformer → LoRA conversion: uses conversion (evidence: `nunchaku/models/transformers/transformer_flux.py`)
- LoRA conversion → LoRA packing: uses packing (evidence: `nunchaku/lora/flux/nunchaku_converter.py`)
- Quantized linear → Quantized ops: calls (evidence: `nunchaku/models/linear.py`)
- PuLID pipeline → Identity encoder: uses encoder (evidence: `nunchaku/pipeline/pipeline_flux_pulid.py`)
- PuLID pipeline → FLUX transformer: uses transformer (evidence: `nunchaku/pipeline/pipeline_flux_pulid.py`)
- FLUX native API → FLUX engine: calls (evidence: `nunchaku/csrc/flux.h`)
- FLUX engine → CUDA kernels: uses kernels (evidence: `src/FluxModel.cpp`)
- FLUX engine → Tensor runtime: uses tensors
