How trufoox/zerollm works
ZeroLLM is a small C++ Transformer language-model learning project. Its principal workflow is preparing text and vocabulary, building model parameters, training with explicit forward/backward passes and per-sequence updates, then producing token predictions. The README documents causal attention, loss, gradient clipping, shared multithreaded weights, and loss output. The sampled training source confirms sequence-level computation and backpropagation; the source index shows its dependencies. Entry-point wiring and several internal calls are not shown, so those links are omitted or marked unverified.
Main entry points: train.cpp Sequence training loop · Source.cpp Application entry · doMath.h Matrix math helpers · normalizer.cpp Text normalization
Components 12
Text and training
- Vocabulary and encoding
Toy LLM/train.cpp - Sequence training loop
Toy LLM/train.cpp - Token prediction
Toy LLM/train.cpp
Transformer computation
- Causal attention
Toy LLM/train.cpp - Softmax and loss
Toy LLM/train.cpp - Manual backpropagation
Toy LLM/train.cpp - Matrix math helpers
Toy LLM/doMath.h
Runtime support
- Text normalization
Toy LLM/normalizer.cpp - Weight file storage
Toy LLM/IO.cpp
Other components
- User
- Application entry
Toy LLM/Source.cpp - Training text
Connections 10
- User → Application entry: starts
- Application entry → Sequence training loop: starts workflow
- Training text → Vocabulary and encoding: provides text
- Sequence training loop → Matrix math helpers: uses helpers
Toy LLM/train.cpp - Sequence training loop → Text normalization: imports
Toy LLM/train.cpp - Sequence training loop → Weight file storage: uses persistence
Toy LLM/train.cpp - Sequence training loop → Causal attention: computes
Toy LLM/train.cpp - Sequence training loop → Softmax and loss: computes
Toy LLM/train.cpp - Sequence training loop → Manual backpropagation: updates with
Toy LLM/train.cpp - Sequence training loop → Token prediction: supports prediction
Toy LLM/train.cpp