# marcfs31/small-llm-in-c architecture

> Architecture diagram of the GitHub repository marcfs31/small-llm-in-c, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-11-14). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/marcfs31/small-llm-in-c
- Repository: https://github.com/marcfs31/small-llm-in-c

## Overview

small-llm-in-c is a CPU inference engine for Qwen3.5 and Gemma 4, accepting either bf16 safetensors or quantized q8 weights. The main workflow is CLI input → tokenization and chat formatting → token-by-token model inference → sampled, decoded response; separate commands run token IDs directly or convert weights to q8. The graph preserves both model families, shared numeric and weight-loading layers, and the documented quantization capability. Edges reflect shown calls or documented responsibilities; unsampled implementations are represented only where the index or README supports them.

Main entry points: [`src/model/lm_model.c`](https://github.com/marcfs31/small-llm-in-c/blob/main/src/model/lm_model.c) (Qwen3.5 model), [`src/model/lm_gemma.c`](https://github.com/marcfs31/small-llm-in-c/blob/main/src/model/lm_gemma.c) (Gemma 4 model), [`src/io/lm_quantfile.c`](https://github.com/marcfs31/small-llm-in-c/blob/main/src/io/lm_quantfile.c) (Q8 weight files), [`src/chat/lm_chat.c`](https://github.com/marcfs31/small-llm-in-c/blob/main/src/chat/lm_chat.c) (Chat generation)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_interaction["Text interaction"]
  node_chat_cli["Chat CLI<br/>[lm-chat.c]"]
  node_chat_loop["Chat generation<br/>[lm_chat.c]"]
  node_templates["Chat templates<br/>[lm_template.c]"]
  node_profiles["Model profiles<br/>[lm_profile.c]"]
  node_sampler["Token sampler<br/>[lm_sampler.c]"]
  node_tokenizer["Tokenizer<br/>[lm_tok.c]"]
  node_normalizer["Unicode normalization<br/>[lm_nfc.c]"]
  node_run_cli["Token-ID runner<br/>[lm-run.c]"]
end

subgraph group_inference["Model inference"]
  node_api["Inference API<br/>[lm.h]"]
  node_qwen_model["Qwen3.5 model<br/>[lm_model.c]"]
  node_gemma_model["Gemma 4 model<br/>[lm_gemma.c]"]
  node_qwen_config["Qwen configuration<br/>[lm_config.c]"]
  node_gemma_config["Gemma configuration<br/>[lm_gemma_config.c]"]
  node_layers["Layer definitions<br/>[lm_layers.c]"]
end

subgraph group_numeric["Numeric kernels"]
  node_ops["Model operations<br/>[lm_ops.c]"]
  node_kernels["Matrix kernels<br/>[lm_kernels.c]"]
  node_quant_kernels["Quantization math<br/>[lm_quant.c]"]
end

subgraph group_weights["Weight handling"]
  node_binding["Tensor binding<br/>[lm_bind.c]"]
  node_safetensors["Safetensors reader<br/>[lm_st.c]"]
  node_quant_cli["Quantization CLI<br/>[lm-quant.c]"]
  node_quantfile["Q8 weight files<br/>[lm_quantfile.c]"]
  node_writer["Safetensors writer<br/>[lm_stw.c]"]
end

node_user(("User"))

node_user -->|"provides prompt"| node_chat_cli
node_chat_cli -->|"starts chat"| node_chat_loop
node_chat_cli -->|"loads profile"| node_profiles
node_chat_cli -->|"selects template"| node_templates
node_chat_loop -->|"feeds tokens"| node_api
node_chat_loop -->|"renders and parses"| node_templates
node_chat_loop -->|"samples tokens"| node_sampler
node_chat_loop -->|"encodes and decodes"| node_tokenizer
node_templates -->|"uses token IDs"| node_tokenizer
node_tokenizer -->|"normalizes text"| node_normalizer
node_run_cli -->|"feeds token IDs"| node_api
node_api -->|"dispatches inference"| node_qwen_model
node_qwen_model -->|"dispatches Gemma"| node_gemma_model
node_qwen_model -->|"loads configuration"| node_qwen_config
node_qwen_model -->|"binds tensors"| node_binding
node_qwen_model -->|"runs model operations"| node_ops
node_qwen_model -->|"computes projections"| node_kernels
node_gemma_model -->|"loads configuration"| node_gemma_config
node_gemma_model -->|"binds tensors"| node_binding
node_gemma_model -->|"runs model operations"| node_ops
node_gemma_model -->|"computes projections"| node_kernels
node_qwen_model -->|"uses layer definitions"| node_layers
node_gemma_model -->|"uses layer definitions"| node_layers
node_binding -->|"reads checkpoint tensors"| node_safetensors
node_quant_cli -->|"writes q8 weights"| node_quantfile
node_quantfile -->|"reads source weights"| node_safetensors
node_quantfile -->|"writes converted tensors"| node_writer
node_quantfile -->|"quantizes weights"| node_quant_kernels
node_quantfile -.->|"supplies q8 weights"| node_qwen_model
node_quantfile -.->|"supplies q8 weights"| node_gemma_model
node_ops -->|"uses matrix kernels"| node_kernels

click node_chat_cli "https://github.com/marcfs31/small-llm-in-c/blob/main/src/cli/lm-chat.c"
click node_chat_loop "https://github.com/marcfs31/small-llm-in-c/blob/main/src/chat/lm_chat.c"
click node_templates "https://github.com/marcfs31/small-llm-in-c/blob/main/src/chat/lm_template.c"
click node_profiles "https://github.com/marcfs31/small-llm-in-c/blob/main/src/chat/lm_profile.c"
click node_sampler "https://github.com/marcfs31/small-llm-in-c/blob/main/src/chat/lm_sampler.c"
click node_tokenizer "https://github.com/marcfs31/small-llm-in-c/blob/main/src/tokenizer/lm_tok.c"
click node_normalizer "https://github.com/marcfs31/small-llm-in-c/blob/main/src/tokenizer/lm_nfc.c"
click node_run_cli "https://github.com/marcfs31/small-llm-in-c/blob/main/src/cli/lm-run.c"
click node_api "https://github.com/marcfs31/small-llm-in-c/blob/main/include/lm/lm.h"
click node_qwen_model "https://github.com/marcfs31/small-llm-in-c/blob/main/src/model/lm_model.c"
click node_gemma_model "https://github.com/marcfs31/small-llm-in-c/blob/main/src/model/lm_gemma.c"
click node_qwen_config "https://github.com/marcfs31/small-llm-in-c/blob/main/src/model/lm_config.c"
click node_gemma_config "https://github.com/marcfs31/small-llm-in-c/blob/main/src/model/lm_gemma_config.c"
click node_layers "https://github.com/marcfs31/small-llm-in-c/blob/main/src/model/lm_layers.c"
click node_binding "https://github.com/marcfs31/small-llm-in-c/blob/main/src/model/lm_bind.c"
click node_safetensors "https://github.com/marcfs31/small-llm-in-c/blob/main/src/io/lm_st.c"
click node_quant_cli "https://github.com/marcfs31/small-llm-in-c/blob/main/src/cli/lm-quant.c"
click node_quantfile "https://github.com/marcfs31/small-llm-in-c/blob/main/src/io/lm_quantfile.c"
click node_writer "https://github.com/marcfs31/small-llm-in-c/blob/main/src/io/lm_stw.c"
click node_ops "https://github.com/marcfs31/small-llm-in-c/blob/main/src/core/lm_ops.c"
click node_kernels "https://github.com/marcfs31/small-llm-in-c/blob/main/src/core/lm_kernels.c"
click node_quant_kernels "https://github.com/marcfs31/small-llm-in-c/blob/main/src/core/lm_quant.c"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_chat_cli,node_chat_loop,node_templates,node_profiles,node_sampler,node_tokenizer,node_normalizer,node_run_cli,node_user toneBlue
class node_api,node_qwen_model,node_gemma_model,node_qwen_config,node_gemma_config,node_layers toneAmber
class node_ops,node_kernels,node_quant_kernels toneMint
class node_binding,node_safetensors,node_quant_cli,node_quantfile,node_writer toneRose
```

## Components

### Text interaction

- [Chat CLI](https://github.com/marcfs31/small-llm-in-c/blob/main/src/cli/lm-chat.c): `src/cli/lm-chat.c`
- [Chat generation](https://github.com/marcfs31/small-llm-in-c/blob/main/src/chat/lm_chat.c): `src/chat/lm_chat.c`
- [Chat templates](https://github.com/marcfs31/small-llm-in-c/blob/main/src/chat/lm_template.c): `src/chat/lm_template.c`
- [Model profiles](https://github.com/marcfs31/small-llm-in-c/blob/main/src/chat/lm_profile.c): `src/chat/lm_profile.c`
- [Token sampler](https://github.com/marcfs31/small-llm-in-c/blob/main/src/chat/lm_sampler.c): `src/chat/lm_sampler.c`
- [Tokenizer](https://github.com/marcfs31/small-llm-in-c/blob/main/src/tokenizer/lm_tok.c): `src/tokenizer/lm_tok.c`
- [Unicode normalization](https://github.com/marcfs31/small-llm-in-c/blob/main/src/tokenizer/lm_nfc.c): `src/tokenizer/lm_nfc.c`
- [Token-ID runner](https://github.com/marcfs31/small-llm-in-c/blob/main/src/cli/lm-run.c): `src/cli/lm-run.c`

### Model inference

- [Inference API](https://github.com/marcfs31/small-llm-in-c/blob/main/include/lm/lm.h): `include/lm/lm.h`
- [Qwen3.5 model](https://github.com/marcfs31/small-llm-in-c/blob/main/src/model/lm_model.c): `src/model/lm_model.c`
- [Gemma 4 model](https://github.com/marcfs31/small-llm-in-c/blob/main/src/model/lm_gemma.c): `src/model/lm_gemma.c`
- [Qwen configuration](https://github.com/marcfs31/small-llm-in-c/blob/main/src/model/lm_config.c): `src/model/lm_config.c`
- [Gemma configuration](https://github.com/marcfs31/small-llm-in-c/blob/main/src/model/lm_gemma_config.c): `src/model/lm_gemma_config.c`
- [Layer definitions](https://github.com/marcfs31/small-llm-in-c/blob/main/src/model/lm_layers.c): `src/model/lm_layers.c`

### Numeric kernels

- [Model operations](https://github.com/marcfs31/small-llm-in-c/blob/main/src/core/lm_ops.c): `src/core/lm_ops.c`
- [Matrix kernels](https://github.com/marcfs31/small-llm-in-c/blob/main/src/core/lm_kernels.c): `src/core/lm_kernels.c`
- [Quantization math](https://github.com/marcfs31/small-llm-in-c/blob/main/src/core/lm_quant.c): `src/core/lm_quant.c`

### Weight handling

- [Tensor binding](https://github.com/marcfs31/small-llm-in-c/blob/main/src/model/lm_bind.c): `src/model/lm_bind.c`
- [Safetensors reader](https://github.com/marcfs31/small-llm-in-c/blob/main/src/io/lm_st.c): `src/io/lm_st.c`
- [Quantization CLI](https://github.com/marcfs31/small-llm-in-c/blob/main/src/cli/lm-quant.c): `src/cli/lm-quant.c`
- [Q8 weight files](https://github.com/marcfs31/small-llm-in-c/blob/main/src/io/lm_quantfile.c): `src/io/lm_quantfile.c`
- [Safetensors writer](https://github.com/marcfs31/small-llm-in-c/blob/main/src/io/lm_stw.c): `src/io/lm_stw.c`

### Other components

- **User**

## Connections

- User → Chat CLI: provides prompt (evidence: `README.md`)
- Chat CLI → Chat generation: starts chat (evidence: `src/cli/lm-chat.c`)
- Chat CLI → Model profiles: loads profile (evidence: `src/cli/lm-chat.c`)
- Chat CLI → Chat templates: selects template (evidence: `src/cli/lm-chat.c`)
- Chat generation → Inference API: feeds tokens (evidence: `src/chat/lm_chat.c`)
- Chat generation → Chat templates: renders and parses
- Chat generation → Token sampler: samples tokens
- Chat generation → Tokenizer: encodes and decodes
- Chat templates → Tokenizer: uses token IDs
- Tokenizer → Unicode normalization: normalizes text (evidence: `src/tokenizer/lm_tok.c`)
- Token-ID runner → Inference API: feeds token IDs (evidence: `src/cli/lm-run.c`)
- Inference API → Qwen3.5 model: dispatches inference (evidence: `src/model/lm_model.c`)
- Qwen3.5 model → Gemma 4 model: dispatches Gemma (evidence: `src/model/lm_model.c`)
- Qwen3.5 model → Qwen configuration: loads configuration (evidence: `src/model/lm_model.c`)
- Qwen3.5 model → Tensor binding: binds tensors (evidence: `src/model/lm_model.c`)
- Qwen3.5 model → Model operations: runs model operations (evidence: `src/model/lm_model.c`)
- Qwen3.5 model → Matrix kernels: computes projections (evidence: `src/model/lm_model.c`)
- Gemma 4 model → Gemma configuration: loads configuration (evidence: `src/model/lm_gemma.c`)
- Gemma 4 model → Tensor binding: binds tensors (evidence: `src/model/lm_gemma.c`)
- Gemma 4 model → Model operations: runs model operations (evidence: `src/model/lm_gemma.c`)
- Gemma 4 model → Matrix kernels: computes projections (evidence: `src/model/lm_gemma.c`)
- Qwen3.5 model → Layer definitions: uses layer definitions (evidence: `src/model/lm_model.c`)
- Gemma 4 model → Layer definitions: uses layer definitions (evidence: `src/model/lm_gemma.c`)
- Tensor binding → Safetensors reader: reads checkpoint tensors
- Quantization CLI → Q8 weight files: writes q8 weights (evidence: `src/cli/lm-quant.c`)
- Q8 weight files → Safetensors reader: reads source weights (evidence: `src/io/lm_quantfile.c`)
- Q8 weight files → Safetensors writer: writes converted tensors (evidence: `src/io/lm_quantfile.c`)
- Q8 weight files → Quantization math: quantizes weights (evidence: `src/io/lm_quantfile.c`)
- Q8 weight files → Qwen3.5 model: supplies q8 weights (evidence: `README.md`)
- Q8 weight files → Gemma 4 model: supplies q8 weights (evidence: `README.md`)
- Model operations → Matrix kernels: uses matrix kernels