# trufoox/zerollm architecture

> Architecture diagram of the GitHub repository trufoox/zerollm, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-10-02). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/trufoox/zerollm
- Repository: https://github.com/trufoox/zerollm

## Overview

ZeroLLM is a small C++ Transformer language-model learning project. Its principal workflow is preparing text and vocabulary, building model parameters, training with explicit forward/backward passes and per-sequence updates, then producing token predictions. The README documents causal attention, loss, gradient clipping, shared multithreaded weights, and loss output. The sampled training source confirms sequence-level computation and backpropagation; the source index shows its dependencies. Entry-point wiring and several internal calls are not shown, so those links are omitted or marked unverified.

Main entry points: [`Toy LLM/train.cpp`](https://github.com/trufoox/zerollm/blob/master/Toy%20LLM/train.cpp) (Sequence training loop), [`Toy LLM/Source.cpp`](https://github.com/trufoox/zerollm/blob/master/Toy%20LLM/Source.cpp) (Application entry), [`Toy LLM/doMath.h`](https://github.com/trufoox/zerollm/blob/master/Toy%20LLM/doMath.h) (Matrix math helpers), [`Toy LLM/normalizer.cpp`](https://github.com/trufoox/zerollm/blob/master/Toy%20LLM/normalizer.cpp) (Text normalization)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_workflow["Text and training"]
  node_vocab["Vocabulary and encoding<br/>[train.cpp]"]
  node_training["Sequence training loop<br/>[train.cpp]"]
  node_inference["Token prediction<br/>[train.cpp]"]
end

subgraph group_model["Transformer computation"]
  node_attention["Causal attention<br/>[train.cpp]"]
  node_loss["Softmax and loss<br/>[train.cpp]"]
  node_backprop["Manual backpropagation<br/>[train.cpp]"]
  node_math["Matrix math helpers<br/>[doMath.h]"]
end

subgraph group_runtime["Runtime support"]
  node_normalizer["Text normalization<br/>[normalizer.cpp]"]
  node_weightfiles[("Weight file storage<br/>[IO.cpp]")]
end

node_user(("User"))
node_source["Application entry<br/>[Source.cpp]"]
node_dataset["Training text"]

node_user -.->|"starts"| node_source
node_source -.->|"starts workflow"| node_training
node_dataset -.->|"provides text"| node_vocab
node_training -->|"uses helpers"| node_math
node_training -->|"imports"| node_normalizer
node_training -->|"uses persistence"| node_weightfiles
node_training -.->|"computes"| node_attention
node_training -.->|"computes"| node_loss
node_training -.->|"updates with"| node_backprop
node_training -.->|"supports prediction"| node_inference

click node_source "https://github.com/trufoox/zerollm/blob/master/Toy%20LLM/Source.cpp"
click node_vocab "https://github.com/trufoox/zerollm/blob/master/Toy%20LLM/train.cpp"
click node_training "https://github.com/trufoox/zerollm/blob/master/Toy%20LLM/train.cpp"
click node_inference "https://github.com/trufoox/zerollm/blob/master/Toy%20LLM/train.cpp"
click node_attention "https://github.com/trufoox/zerollm/blob/master/Toy%20LLM/train.cpp"
click node_loss "https://github.com/trufoox/zerollm/blob/master/Toy%20LLM/train.cpp"
click node_backprop "https://github.com/trufoox/zerollm/blob/master/Toy%20LLM/train.cpp"
click node_math "https://github.com/trufoox/zerollm/blob/master/Toy%20LLM/doMath.h"
click node_normalizer "https://github.com/trufoox/zerollm/blob/master/Toy%20LLM/normalizer.cpp"
click node_weightfiles "https://github.com/trufoox/zerollm/blob/master/Toy%20LLM/IO.cpp"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_vocab,node_training,node_inference,node_user toneBlue
class node_attention,node_loss,node_backprop,node_math toneAmber
class node_normalizer,node_weightfiles toneMint
class node_source toneTeal
class node_dataset toneIndigo
```

## Components

### Text and training

- [Vocabulary and encoding](https://github.com/trufoox/zerollm/blob/master/Toy%20LLM/train.cpp): `Toy LLM/train.cpp`
- [Sequence training loop](https://github.com/trufoox/zerollm/blob/master/Toy%20LLM/train.cpp): `Toy LLM/train.cpp`
- [Token prediction](https://github.com/trufoox/zerollm/blob/master/Toy%20LLM/train.cpp): `Toy LLM/train.cpp`

### Transformer computation

- [Causal attention](https://github.com/trufoox/zerollm/blob/master/Toy%20LLM/train.cpp): `Toy LLM/train.cpp`
- [Softmax and loss](https://github.com/trufoox/zerollm/blob/master/Toy%20LLM/train.cpp): `Toy LLM/train.cpp`
- [Manual backpropagation](https://github.com/trufoox/zerollm/blob/master/Toy%20LLM/train.cpp): `Toy LLM/train.cpp`
- [Matrix math helpers](https://github.com/trufoox/zerollm/blob/master/Toy%20LLM/doMath.h): `Toy LLM/doMath.h`

### Runtime support

- [Text normalization](https://github.com/trufoox/zerollm/blob/master/Toy%20LLM/normalizer.cpp): `Toy LLM/normalizer.cpp`
- [Weight file storage](https://github.com/trufoox/zerollm/blob/master/Toy%20LLM/IO.cpp): `Toy LLM/IO.cpp`

### Other components

- **User**
- [Application entry](https://github.com/trufoox/zerollm/blob/master/Toy%20LLM/Source.cpp): `Toy LLM/Source.cpp`
- **Training text**

## Connections

- User → Application entry: starts
- Application entry → Sequence training loop: starts workflow
- Training text → Vocabulary and encoding: provides text
- Sequence training loop → Matrix math helpers: uses helpers (evidence: `Toy LLM/train.cpp`)
- Sequence training loop → Text normalization: imports (evidence: `Toy LLM/train.cpp`)
- Sequence training loop → Weight file storage: uses persistence (evidence: `Toy LLM/train.cpp`)
- Sequence training loop → Causal attention: computes (evidence: `Toy LLM/train.cpp`)
- Sequence training loop → Softmax and loss: computes (evidence: `Toy LLM/train.cpp`)
- Sequence training loop → Manual backpropagation: updates with (evidence: `Toy LLM/train.cpp`)
- Sequence training loop → Token prediction: supports prediction (evidence: `Toy LLM/train.cpp`)
