How mit-han-lab/nunchaku works
Nunchaku is a GPU inference engine for quantized neural networks, particularly image-generation models. A typical user-facing workflow is to launch a model-specific demo, run inference through a Nunchaku transformer, and receive generated images; the repository also provides LoRA conversion, caching, and PuLID personalization. The native C++/CUDA engine and Python model layers implement the inference core. Edges are limited to relationships shown in excerpts, the source index, or README statements; several listed model and demo files were not sampled, so their internal wiring is not inferred.
Main entry points: run_gradio.py FLUX demos · FluxModel.cpp FLUX engine · transformer_flux.py FLUX transformer · pipeline_flux_pulid.py PuLID pipeline
Components 17
Demos and workflows
- FLUX demos
app/flux.1/depth_canny/run_gradio.py - Sana demo
app/sana/t2i/run_gradio.py
Model inference
- FLUX transformer
nunchaku/models/transformers/transformer_flux.py - Other transformers
nunchaku/models/transformers/transformer_qwenimage.py - Quantized linear
nunchaku/models/linear.py
Inference features
- LoRA conversion
nunchaku/lora/flux/nunchaku_converter.py - LoRA packing
nunchaku/lora/flux/packer.py - Inference caching
nunchaku/caching/fbcache.py - PuLID pipeline
nunchaku/pipeline/pipeline_flux_pulid.py - Identity encoder
nunchaku/models/pulid/encoders_transformer.py - Safety checker
nunchaku/models/safety_checker.py
Native engine
- Quantized ops
nunchaku/ops/gemm.py - FLUX native API
nunchaku/csrc/flux.h - FLUX engine
src/FluxModel.cpp - Tensor runtime
src/Tensor.h - CUDA kernels
src/kernels/zgemm/zgemm.h
Other components
- User
Connections 13
- User → FLUX demos: launches
- User → Sana demo: launches
- FLUX demos → FLUX transformer: uses
app/flux.1/depth_canny/run_gradio.py - FLUX demos → Safety checker: checks output
app/flux.1/depth_canny/run_gradio.py - Sana demo → Safety checker: checks output
app/sana/t2i/run_gradio.py - FLUX transformer → LoRA conversion: uses conversion
nunchaku/models/transformers/transformer_flux.py - LoRA conversion → LoRA packing: uses packing
nunchaku/lora/flux/nunchaku_converter.py - Quantized linear → Quantized ops: calls
nunchaku/models/linear.py - PuLID pipeline → Identity encoder: uses encoder
nunchaku/pipeline/pipeline_flux_pulid.py - PuLID pipeline → FLUX transformer: uses transformer
nunchaku/pipeline/pipeline_flux_pulid.py - FLUX native API → FLUX engine: calls
nunchaku/csrc/flux.h - FLUX engine → CUDA kernels: uses kernels
src/FluxModel.cpp - FLUX engine → Tensor runtime: uses tensors