marcfs31/small-llm-in-c

How marcfs31/small-llm-in-c works

small-llm-in-c is a CPU inference engine for Qwen3.5 and Gemma 4, accepting either bf16 safetensors or quantized q8 weights. The main workflow is CLI input → tokenization and chat formatting → token-by-token model inference → sampled, decoded response; separate commands run token IDs directly or convert weights to q8. The graph preserves both model families, shared numeric and weight-loading layers, and the documented quantization capability. Edges reflect shown calls or documented responsibilities; unsampled implementations are represented only where the index or README supports them.

Main entry points: lm_model.c Qwen3.5 model · lm_gemma.c Gemma 4 model · lm_quantfile.c Q8 weight files · lm_chat.c Chat generation

Components 23

Text interaction

Chat CLI src/cli/lm-chat.c
Chat generation src/chat/lm_chat.c
Chat templates src/chat/lm_template.c
Model profiles src/chat/lm_profile.c
Token sampler src/chat/lm_sampler.c
Tokenizer src/tokenizer/lm_tok.c
Unicode normalization src/tokenizer/lm_nfc.c
Token-ID runner src/cli/lm-run.c

Model inference

Inference API include/lm/lm.h
Qwen3.5 model src/model/lm_model.c
Gemma 4 model src/model/lm_gemma.c
Qwen configuration src/model/lm_config.c
Gemma configuration src/model/lm_gemma_config.c
Layer definitions src/model/lm_layers.c

Numeric kernels

Model operations src/core/lm_ops.c
Matrix kernels src/core/lm_kernels.c
Quantization math src/core/lm_quant.c

Weight handling

Tensor binding src/model/lm_bind.c
Safetensors reader src/io/lm_st.c
Quantization CLI src/cli/lm-quant.c
Q8 weight files src/io/lm_quantfile.c
Safetensors writer src/io/lm_stw.c

Other components

User

Connections 31

User → Chat CLI: provides prompt README.md
Chat CLI → Chat generation: starts chat src/cli/lm-chat.c
Chat CLI → Model profiles: loads profile src/cli/lm-chat.c
Chat CLI → Chat templates: selects template src/cli/lm-chat.c
Chat generation → Inference API: feeds tokens src/chat/lm_chat.c
Chat generation → Chat templates: renders and parses
Chat generation → Token sampler: samples tokens
Chat generation → Tokenizer: encodes and decodes
Chat templates → Tokenizer: uses token IDs
Tokenizer → Unicode normalization: normalizes text src/tokenizer/lm_tok.c
Token-ID runner → Inference API: feeds token IDs src/cli/lm-run.c
Inference API → Qwen3.5 model: dispatches inference src/model/lm_model.c
Qwen3.5 model → Gemma 4 model: dispatches Gemma src/model/lm_model.c
Qwen3.5 model → Qwen configuration: loads configuration src/model/lm_model.c
Qwen3.5 model → Tensor binding: binds tensors src/model/lm_model.c
Qwen3.5 model → Model operations: runs model operations src/model/lm_model.c
Qwen3.5 model → Matrix kernels: computes projections src/model/lm_model.c
Gemma 4 model → Gemma configuration: loads configuration src/model/lm_gemma.c
Gemma 4 model → Tensor binding: binds tensors src/model/lm_gemma.c
Gemma 4 model → Model operations: runs model operations src/model/lm_gemma.c
Gemma 4 model → Matrix kernels: computes projections src/model/lm_gemma.c
Qwen3.5 model → Layer definitions: uses layer definitions src/model/lm_model.c
Gemma 4 model → Layer definitions: uses layer definitions src/model/lm_gemma.c
Tensor binding → Safetensors reader: reads checkpoint tensors
Quantization CLI → Q8 weight files: writes q8 weights src/cli/lm-quant.c
Q8 weight files → Safetensors reader: reads source weights src/io/lm_quantfile.c
Q8 weight files → Safetensors writer: writes converted tensors src/io/lm_quantfile.c
Q8 weight files → Quantization math: quantizes weights src/io/lm_quantfile.c
Q8 weight files → Qwen3.5 model: supplies q8 weights README.md
Q8 weight files → Gemma 4 model: supplies q8 weights README.md
Model operations → Matrix kernels: uses matrix kernels

Drawn by GitDiagram on November 14, 2026 from the repository's file tree, README and source files. Read as Markdown ·