How microsoft/bitnet works
bitnet.cpp is an inference framework for ternary/1-bit language and embedding models, with optimized CPU kernels and a separate GPU inference path. The user workflow is to prepare a compatible model, then run tokenized prompts through model inference and receive generated output; CPU inference is integrated with the bundled llama.cpp tree, while GPU generation has its own Python runtime. The graph preserves model conversion, embedding support, and CPU kernel variants. Some runtime wiring is unsampled, so uncertain links are omitted rather than inferred.
Main entry points: generate.py GPU Generation · ggml-bitnet-mad.cpp BitNet Operations · ggml-bitnet-lut.cpp LUT Operations · convert_checkpoint.py GPU Checkpoint Prep
Components 17
Model Preparation
- HF Conversion
utils/convert-hf-to-gguf-bitnet.py - Conversion Tools
utils - Embedding Conversion
utils/convert-bitnet-embedding-to-gguf.py
CPU Inference
- CPU Runtime
3rdparty/llama.cpp - BitNet Operations
src/ggml-bitnet-mad.cpp - LUT Operations
src/ggml-bitnet-lut.cpp - BitNet API
include/ggml-bitnet.h
GPU Inference
- GPU Checkpoint Prep
gpu/convert_checkpoint.py - Weight Packing
gpu/pack_weight.py - GPU Transformer
gpu/model.py - GPU Generation
gpu/generate.py - Tokenizer and Chat
gpu/tokenizer.py - Sampling
gpu/sample_utils.py - Generation Stats
gpu/stats.py
Kernel Implementations
- LUT Kernels
include/bitnet-lut-kernels.h - Preset Kernels
preset_kernels
Other components
- Model User
Connections 15
- Model User → HF Conversion: prepares model
README.md - Model User → CPU Runtime: runs inference
README.md - Model User → GPU Generation: prompts generation
gpu/generate.py - HF Conversion → Conversion Tools: uses conversion
utils/convert-hf-to-gguf-bitnet.py - GPU Checkpoint Prep → Weight Packing: packs weights
gpu/convert_checkpoint.py - GPU Checkpoint Prep → GPU Transformer: uses model
gpu/convert_checkpoint.py - GPU Generation → GPU Transformer: runs transformer
gpu/generate.py - GPU Generation → Tokenizer and Chat: encodes prompts
gpu/generate.py - GPU Generation → Sampling: uses sampling
gpu/generate.py - GPU Generation → Generation Stats: records stats
gpu/generate.py - BitNet Operations → BitNet API: uses API
src/ggml-bitnet-mad.cpp - LUT Operations → BitNet API: uses API
src/ggml-bitnet-lut.cpp - LUT Operations → LUT Kernels: includes kernels
src/ggml-bitnet-lut.cpp - BitNet Operations → Preset Kernels: selects variants
- Model User → Embedding Conversion: converts embeddings
README.md