# matrixsmaster/ligguf architecture

> Architecture diagram of the GitHub repository matrixsmaster/ligguf, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-10-05). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/matrixsmaster/ligguf
- Repository: https://github.com/matrixsmaster/ligguf

## Overview

LiGGUF is a compact local GGUF inference engine with two single-file reference editions and a modular FAST runtime. The main user workflow is to load a model, tokenize a prompt, run transformer inference, sample and print generated tokens; FAST also supports interactive chat and optional retrieval-augmented memory. The graph includes documented model-family support, CPU acceleration, visualization, and the experimental distributed example. Some FAST internals were only partially excerpted, so uncertain links are omitted or marked unverified.

Main entry points: [`fast/ligguf_fast_qk.cpp`](https://github.com/matrixsmaster/ligguf/blob/master/fast/ligguf_fast_qk.cpp) (FAST Runtime), [`cpp/ligguf_distrib.cpp`](https://github.com/matrixsmaster/ligguf/blob/master/cpp/ligguf_distrib.cpp) (SARA Example), [`fast/vdb.cpp`](https://github.com/matrixsmaster/ligguf/blob/master/fast/vdb.cpp) (BERT Embeddings), [`fast/lil_math.cpp`](https://github.com/matrixsmaster/ligguf/blob/master/fast/lil_math.cpp) (Tensor Math)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_reference["Reference Engines"]
  node_c_engine["C Reference<br/>[ligguf.c]"]
  node_cpp_engine["C++ Reference<br/>[ligguf.cpp]"]
  node_dist_example["SARA Example<br/>[ligguf_distrib.cpp]"]
end

subgraph group_runtime["FAST Runtime"]
  node_fast_cli["FAST Runtime<br/>[ligguf_fast_qk.cpp]"]
  node_gguf["GGUF Loader<br/>[lil_gguf.cpp]"]
  node_tokenizer["Tokenizer<br/>[tokenize.cpp]"]
  node_chat["Chat and State<br/>[ligguf_fast_qk.cpp]"]
end

subgraph group_memory["Retrieval Memory"]
  node_bert_memory["BERT Embeddings<br/>[vdb.cpp]"]
  node_vector_store[("Vector Retrieval<br/>[vdb.cpp]")]
end

subgraph group_compute["Inference Compute"]
  node_math["Tensor Math<br/>[lil_math.cpp]"]
  node_avx["AVX2 Kernels<br/>[arch_avx2.h]"]
  node_neon["NEON Kernels<br/>[arch_neon.h]"]
end

subgraph group_extensions["Runtime Extensions"]
  node_visualizer["Hotpath Visualizer<br/>[viz_sdl.cpp]"]
end

node_user(("CLI User"))
node_gguf_model["GGUF Model"]
node_sdl["SDL2"]
node_workers(("Peer Workers"))

node_user -->|"runs"| node_fast_cli
node_user -->|"runs"| node_c_engine
node_user -->|"runs"| node_cpp_engine
node_fast_cli -->|"loads model"| node_gguf
node_gguf_model -->|"provides weights"| node_gguf
node_fast_cli -->|"tokenizes prompt"| node_tokenizer
node_fast_cli -->|"runs inference"| node_math
node_math -->|"uses kernels"| node_avx
node_math -->|"uses kernels"| node_neon
node_fast_cli -->|"handles chat"| node_chat
node_chat -.->|"feeds turns"| node_fast_cli
node_fast_cli -->|"requests retrieval"| node_bert_memory
node_bert_memory -->|"loads BERT model"| node_gguf
node_bert_memory -->|"embeds and queries"| node_vector_store
node_vector_store -->|"supplies prompt memory"| node_fast_cli
node_fast_cli -->|"traces activations"| node_visualizer
node_visualizer -->|"renders with"| node_sdl
node_user -->|"runs workers"| node_dist_example
node_dist_example -->|"exchanges activations"| node_workers
node_dist_example -->|"loads shared model"| node_gguf_model

click node_c_engine "https://github.com/matrixsmaster/ligguf/blob/master/c/ligguf.c"
click node_cpp_engine "https://github.com/matrixsmaster/ligguf/blob/master/cpp/ligguf.cpp"
click node_dist_example "https://github.com/matrixsmaster/ligguf/blob/master/cpp/ligguf_distrib.cpp"
click node_fast_cli "https://github.com/matrixsmaster/ligguf/blob/master/fast/ligguf_fast_qk.cpp"
click node_gguf "https://github.com/matrixsmaster/ligguf/blob/master/fast/lil_gguf.cpp"
click node_tokenizer "https://github.com/matrixsmaster/ligguf/blob/master/fast/tokenize.cpp"
click node_chat "https://github.com/matrixsmaster/ligguf/blob/master/fast/ligguf_fast_qk.cpp"
click node_math "https://github.com/matrixsmaster/ligguf/blob/master/fast/lil_math.cpp"
click node_avx "https://github.com/matrixsmaster/ligguf/blob/master/fast/arch_avx2.h"
click node_neon "https://github.com/matrixsmaster/ligguf/blob/master/fast/arch_neon.h"
click node_bert_memory "https://github.com/matrixsmaster/ligguf/blob/master/fast/vdb.cpp"
click node_vector_store "https://github.com/matrixsmaster/ligguf/blob/master/fast/vdb.cpp"
click node_visualizer "https://github.com/matrixsmaster/ligguf/blob/master/fast/viz_sdl.cpp"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_c_engine,node_cpp_engine,node_dist_example,node_user toneBlue
class node_fast_cli,node_gguf,node_tokenizer,node_chat toneAmber
class node_bert_memory,node_vector_store toneMint
class node_math,node_avx,node_neon,node_workers toneRose
class node_visualizer,node_gguf_model,node_sdl toneIndigo
```

## Components

### Reference Engines

- [C Reference](https://github.com/matrixsmaster/ligguf/blob/master/c/ligguf.c): `c/ligguf.c`
- [C++ Reference](https://github.com/matrixsmaster/ligguf/blob/master/cpp/ligguf.cpp): `cpp/ligguf.cpp`
- [SARA Example](https://github.com/matrixsmaster/ligguf/blob/master/cpp/ligguf_distrib.cpp): `cpp/ligguf_distrib.cpp`

### FAST Runtime

- [FAST Runtime](https://github.com/matrixsmaster/ligguf/blob/master/fast/ligguf_fast_qk.cpp): `fast/ligguf_fast_qk.cpp`
- [GGUF Loader](https://github.com/matrixsmaster/ligguf/blob/master/fast/lil_gguf.cpp): `fast/lil_gguf.cpp`
- [Tokenizer](https://github.com/matrixsmaster/ligguf/blob/master/fast/tokenize.cpp): `fast/tokenize.cpp`
- [Chat and State](https://github.com/matrixsmaster/ligguf/blob/master/fast/ligguf_fast_qk.cpp): `fast/ligguf_fast_qk.cpp`

### Retrieval Memory

- [BERT Embeddings](https://github.com/matrixsmaster/ligguf/blob/master/fast/vdb.cpp): `fast/vdb.cpp`
- [Vector Retrieval](https://github.com/matrixsmaster/ligguf/blob/master/fast/vdb.cpp): `fast/vdb.cpp`

### Inference Compute

- [Tensor Math](https://github.com/matrixsmaster/ligguf/blob/master/fast/lil_math.cpp): `fast/lil_math.cpp`
- [AVX2 Kernels](https://github.com/matrixsmaster/ligguf/blob/master/fast/arch_avx2.h): `fast/arch_avx2.h`
- [NEON Kernels](https://github.com/matrixsmaster/ligguf/blob/master/fast/arch_neon.h): `fast/arch_neon.h`

### Runtime Extensions

- [Hotpath Visualizer](https://github.com/matrixsmaster/ligguf/blob/master/fast/viz_sdl.cpp): `fast/viz_sdl.cpp`

### Other components

- **CLI User**
- **GGUF Model**
- **SDL2**
- **Peer Workers**

## Connections

- CLI User → FAST Runtime: runs (evidence: `README.md`)
- CLI User → C Reference: runs (evidence: `README.md`)
- CLI User → C++ Reference: runs (evidence: `README.md`)
- FAST Runtime → GGUF Loader: loads model (evidence: `fast/ligguf_fast_qk.cpp`)
- GGUF Model → GGUF Loader: provides weights (evidence: `README.md`)
- FAST Runtime → Tokenizer: tokenizes prompt (evidence: `fast/ligguf_fast_qk.cpp`)
- FAST Runtime → Tensor Math: runs inference (evidence: `fast/ligguf_fast_qk.cpp`)
- Tensor Math → AVX2 Kernels: uses kernels (evidence: `fast/lil_math.cpp`)
- Tensor Math → NEON Kernels: uses kernels (evidence: `fast/lil_math.cpp`)
- FAST Runtime → Chat and State: handles chat (evidence: `README.md`)
- Chat and State → FAST Runtime: feeds turns (evidence: `fast/ligguf_fast_qk.cpp`)
- FAST Runtime → BERT Embeddings: requests retrieval (evidence: `README.md`)
- BERT Embeddings → GGUF Loader: loads BERT model (evidence: `fast/vdb.cpp`)
- BERT Embeddings → Vector Retrieval: embeds and queries (evidence: `fast/vdb.cpp`)
- Vector Retrieval → FAST Runtime: supplies prompt memory (evidence: `README.md`)
- FAST Runtime → Hotpath Visualizer: traces activations (evidence: `fast/ligguf_fast_qk.cpp`)
- Hotpath Visualizer → SDL2: renders with (evidence: `README.md`)
- CLI User → SARA Example: runs workers (evidence: `README.md`)
- SARA Example → Peer Workers: exchanges activations (evidence: `README.md`)
- SARA Example → GGUF Model: loads shared model (evidence: `README.md`)
