How gerard-payne/mini-gpt works
MiniGPT-JS is an educational decoder-only transformer library with a browser demo. Its main workflow encodes corpus text, prepares next-token examples, trains a model, then generates and decodes text. The README also documents character and BPE tokenization, an alternate full-autograd model, and selectable browser compute backends. Demo wiring and some subsystem internals were not sampled, so only documented capabilities are shown without speculative implementation links.
Main entry points: train.js Training Utilities · model.js MiniGPT · utils.js Math and Sampling · demo.html Browser Demo
Components 13
Tokenization
- CharTokenizer
src/tokenizer.js - BPE Tokenizer
src/bpe_tokenizer.js
Training
- Training Utilities
src/train.js - Adam and Gradients
src/autograd.js
Model and Inference
- MiniGPT
src/model.js - Autograd MiniGPT
src/model_with_autograd.js - Math and Sampling
src/utils.js
Compute Backends
- WebGL Backend
src/webgl.js - WebGPU Backend
src/webgpu.js
Other components
- Learner
- Text Corpus
- Browser Demo
demo.html - Generated Text
Connections 16
- Learner → Text Corpus: provides text
- Learner → Training Utilities: configures training
- Text Corpus → CharTokenizer: builds vocabulary
- Text Corpus → BPE Tokenizer: trains vocabulary
- Training Utilities → CharTokenizer: encodes data
- Training Utilities → MiniGPT: trains model
- Training Utilities → MiniGPT: evaluates model
- Training Utilities → Math and Sampling: samples batches
- MiniGPT → Math and Sampling: uses math
- MiniGPT → Generated Text: generates tokens
- Learner → Browser Demo: uses demo
- Browser Demo → WebGL Backend: selects backend
- Browser Demo → WebGPU Backend: selects backend
- Autograd MiniGPT → Adam and Gradients: uses gradients
- Autograd MiniGPT → Math and Sampling: uses math
- Adam and Gradients → Math and Sampling: uses operations