How karpathy/mingpt works
minGPT is a compact PyTorch GPT library with training and inference, plus two runnable training examples and notebook demos. Its main workflow turns token-index sequences into next-token predictions, trains the model from dataset batches, and generates or evaluates sequences. The README documents the tokenizer, model, trainer, and demos; source excerpts confirm trainer/model calls and project wiring. Notebook internals and some model details were not sampled, so their relationships are limited to documented capabilities.
Main entry points: model.py GPT Model · adder.py Adder Example · chargpt.py CharGPT Example · trainer.py Training Loop
Components 15
Model Runtime
- BPE Tokenizer
mingpt/bpe.py - GPT Model
mingpt/model.py
Training
- Training Loop
mingpt/trainer.py - Config Utilities
mingpt/utils.py
Examples
- Adder Example
projects/adder/adder.py - Addition Dataset
projects/adder/adder.py - Addition Evaluation
projects/adder/adder.py - CharGPT Example
projects/chargpt/chargpt.py - Character Dataset
projects/chargpt/chargpt.py - Text Sampling
projects/chargpt/chargpt.py - Sorting Demo
demo.ipynb - GPT-2 Generation
generate.ipynb
Model Assets
- Transformers Hub
- BPE Asset Cache
mingpt/bpe.py
Other components
- Developer
Connections 29
- Developer → BPE Tokenizer: tokenizes text
README.md - BPE Tokenizer → GPT Model: produces indices
README.md - Developer → GPT Model: configures model
README.md - Training Loop → GPT Model: calls forward
mingpt/trainer.py - Training Loop → GPT Model: configures optimizer
mingpt/trainer.py - Training Loop → Config Utilities: uses config
mingpt/trainer.py - GPT Model → Config Utilities: uses config
mingpt/model.py - BPE Tokenizer → BPE Asset Cache: reads or downloads
mingpt/bpe.py - BPE Tokenizer → Transformers Hub: downloads GPT-2 assets
mingpt/bpe.py - Developer → Adder Example: runs example
README.md - Adder Example → Addition Dataset: constructs datasets
projects/adder/adder.py - Adder Example → GPT Model: constructs model
projects/adder/adder.py - Adder Example → Training Loop: constructs trainer
projects/adder/adder.py - Adder Example → Config Utilities: uses utilities
projects/adder/adder.py - Adder Example → Addition Evaluation: evaluates predictions
projects/adder/adder.py - Addition Evaluation → GPT Model: generates sums
projects/adder/adder.py - Developer → CharGPT Example: runs example
README.md - CharGPT Example → Character Dataset: constructs dataset
projects/chargpt/chargpt.py - CharGPT Example → GPT Model: constructs model
projects/chargpt/chargpt.py - CharGPT Example → Training Loop: constructs trainer
projects/chargpt/chargpt.py - CharGPT Example → Config Utilities: uses utilities
projects/chargpt/chargpt.py - CharGPT Example → Text Sampling: registers callback
projects/chargpt/chargpt.py - Text Sampling → GPT Model: samples completion
projects/chargpt/chargpt.py - Developer → Sorting Demo: uses notebook
README.md - Sorting Demo → GPT Model: uses model and trainer
README.md - Developer → GPT-2 Generation: uses notebook
README.md - GPT-2 Generation → Transformers Hub: loads pretrained GPT-2
README.md - GPT-2 Generation → GPT Model: generates text
README.md - GPT Model → Transformers Hub: imports pretrained weights
mingpt/model.py