# karpathy/mingpt architecture

> Architecture diagram of the GitHub repository karpathy/mingpt, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-11-09). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/karpathy/mingpt
- Repository: https://github.com/karpathy/mingpt

## Overview

minGPT is a compact PyTorch GPT library with training and inference, plus two runnable training examples and notebook demos. Its main workflow turns token-index sequences into next-token predictions, trains the model from dataset batches, and generates or evaluates sequences. The README documents the tokenizer, model, trainer, and demos; source excerpts confirm trainer/model calls and project wiring. Notebook internals and some model details were not sampled, so their relationships are limited to documented capabilities.

Main entry points: [`mingpt/model.py`](https://github.com/karpathy/mingpt/blob/master/mingpt/model.py) (GPT Model), [`projects/adder/adder.py`](https://github.com/karpathy/mingpt/blob/master/projects/adder/adder.py) (Adder Example), [`projects/chargpt/chargpt.py`](https://github.com/karpathy/mingpt/blob/master/projects/chargpt/chargpt.py) (CharGPT Example), [`mingpt/trainer.py`](https://github.com/karpathy/mingpt/blob/master/mingpt/trainer.py) (Training Loop)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_runtime["Model Runtime"]
  node_bpe["BPE Tokenizer<br/>[bpe.py]"]
  node_gpt["GPT Model<br/>[model.py]"]
end

subgraph group_training["Training"]
  node_trainer["Training Loop<br/>[trainer.py]"]
  node_config["Config Utilities<br/>[utils.py]"]
end

subgraph group_examples["Examples"]
  node_adder["Adder Example<br/>[adder.py]"]
  node_addition["Addition Dataset<br/>[adder.py]"]
  node_addeval["Addition Evaluation<br/>[adder.py]"]
  node_char["CharGPT Example<br/>[chargpt.py]"]
  node_chardata["Character Dataset<br/>[chargpt.py]"]
  node_sampling["Text Sampling<br/>[chargpt.py]"]
  node_demo["Sorting Demo<br/>[demo.ipynb]"]
  node_generation["GPT-2 Generation<br/>[generate.ipynb]"]
end

subgraph group_assets["Model Assets"]
  node_hf["Transformers Hub"]
  node_bpefiles["BPE Asset Cache<br/>[bpe.py]"]
end

node_user(("Developer"))

node_user -->|"tokenizes text"| node_bpe
node_bpe -->|"produces indices"| node_gpt
node_user -->|"configures model"| node_gpt
node_trainer -->|"calls forward"| node_gpt
node_trainer -->|"configures optimizer"| node_gpt
node_trainer -->|"uses config"| node_config
node_gpt -->|"uses config"| node_config
node_bpe -->|"reads or downloads"| node_bpefiles
node_bpe -->|"downloads GPT-2 assets"| node_hf
node_user -->|"runs example"| node_adder
node_adder -->|"constructs datasets"| node_addition
node_adder -->|"constructs model"| node_gpt
node_adder -->|"constructs trainer"| node_trainer
node_adder -->|"uses utilities"| node_config
node_adder -->|"evaluates predictions"| node_addeval
node_addeval -->|"generates sums"| node_gpt
node_user -->|"runs example"| node_char
node_char -->|"constructs dataset"| node_chardata
node_char -->|"constructs model"| node_gpt
node_char -->|"constructs trainer"| node_trainer
node_char -->|"uses utilities"| node_config
node_char -->|"registers callback"| node_sampling
node_sampling -->|"samples completion"| node_gpt
node_user -->|"uses notebook"| node_demo
node_demo -->|"uses model and trainer"| node_gpt
node_user -->|"uses notebook"| node_generation
node_generation -->|"loads pretrained GPT-2"| node_hf
node_generation -->|"generates text"| node_gpt
node_gpt -->|"imports pretrained weights"| node_hf

click node_bpe "https://github.com/karpathy/mingpt/blob/master/mingpt/bpe.py"
click node_gpt "https://github.com/karpathy/mingpt/blob/master/mingpt/model.py"
click node_trainer "https://github.com/karpathy/mingpt/blob/master/mingpt/trainer.py"
click node_config "https://github.com/karpathy/mingpt/blob/master/mingpt/utils.py"
click node_bpefiles "https://github.com/karpathy/mingpt/blob/master/mingpt/bpe.py"
click node_adder "https://github.com/karpathy/mingpt/blob/master/projects/adder/adder.py"
click node_addition "https://github.com/karpathy/mingpt/blob/master/projects/adder/adder.py"
click node_addeval "https://github.com/karpathy/mingpt/blob/master/projects/adder/adder.py"
click node_char "https://github.com/karpathy/mingpt/blob/master/projects/chargpt/chargpt.py"
click node_chardata "https://github.com/karpathy/mingpt/blob/master/projects/chargpt/chargpt.py"
click node_sampling "https://github.com/karpathy/mingpt/blob/master/projects/chargpt/chargpt.py"
click node_demo "https://github.com/karpathy/mingpt/blob/master/demo.ipynb"
click node_generation "https://github.com/karpathy/mingpt/blob/master/generate.ipynb"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_bpe,node_gpt toneBlue
class node_trainer,node_config toneAmber
class node_adder,node_addition,node_addeval,node_char,node_chardata,node_sampling,node_demo,node_generation toneMint
class node_hf,node_bpefiles toneRose
class node_user toneIndigo
```

## Components

### Model Runtime

- [BPE Tokenizer](https://github.com/karpathy/mingpt/blob/master/mingpt/bpe.py): `mingpt/bpe.py`
- [GPT Model](https://github.com/karpathy/mingpt/blob/master/mingpt/model.py): `mingpt/model.py`

### Training

- [Training Loop](https://github.com/karpathy/mingpt/blob/master/mingpt/trainer.py): `mingpt/trainer.py`
- [Config Utilities](https://github.com/karpathy/mingpt/blob/master/mingpt/utils.py): `mingpt/utils.py`

### Examples

- [Adder Example](https://github.com/karpathy/mingpt/blob/master/projects/adder/adder.py): `projects/adder/adder.py`
- [Addition Dataset](https://github.com/karpathy/mingpt/blob/master/projects/adder/adder.py): `projects/adder/adder.py`
- [Addition Evaluation](https://github.com/karpathy/mingpt/blob/master/projects/adder/adder.py): `projects/adder/adder.py`
- [CharGPT Example](https://github.com/karpathy/mingpt/blob/master/projects/chargpt/chargpt.py): `projects/chargpt/chargpt.py`
- [Character Dataset](https://github.com/karpathy/mingpt/blob/master/projects/chargpt/chargpt.py): `projects/chargpt/chargpt.py`
- [Text Sampling](https://github.com/karpathy/mingpt/blob/master/projects/chargpt/chargpt.py): `projects/chargpt/chargpt.py`
- [Sorting Demo](https://github.com/karpathy/mingpt/blob/master/demo.ipynb): `demo.ipynb`
- [GPT-2 Generation](https://github.com/karpathy/mingpt/blob/master/generate.ipynb): `generate.ipynb`

### Model Assets

- **Transformers Hub**
- [BPE Asset Cache](https://github.com/karpathy/mingpt/blob/master/mingpt/bpe.py): `mingpt/bpe.py`

### Other components

- **Developer**

## Connections

- Developer → BPE Tokenizer: tokenizes text (evidence: `README.md`)
- BPE Tokenizer → GPT Model: produces indices (evidence: `README.md`)
- Developer → GPT Model: configures model (evidence: `README.md`)
- Training Loop → GPT Model: calls forward (evidence: `mingpt/trainer.py`)
- Training Loop → GPT Model: configures optimizer (evidence: `mingpt/trainer.py`)
- Training Loop → Config Utilities: uses config (evidence: `mingpt/trainer.py`)
- GPT Model → Config Utilities: uses config (evidence: `mingpt/model.py`)
- BPE Tokenizer → BPE Asset Cache: reads or downloads (evidence: `mingpt/bpe.py`)
- BPE Tokenizer → Transformers Hub: downloads GPT-2 assets (evidence: `mingpt/bpe.py`)
- Developer → Adder Example: runs example (evidence: `README.md`)
- Adder Example → Addition Dataset: constructs datasets (evidence: `projects/adder/adder.py`)
- Adder Example → GPT Model: constructs model (evidence: `projects/adder/adder.py`)
- Adder Example → Training Loop: constructs trainer (evidence: `projects/adder/adder.py`)
- Adder Example → Config Utilities: uses utilities (evidence: `projects/adder/adder.py`)
- Adder Example → Addition Evaluation: evaluates predictions (evidence: `projects/adder/adder.py`)
- Addition Evaluation → GPT Model: generates sums (evidence: `projects/adder/adder.py`)
- Developer → CharGPT Example: runs example (evidence: `README.md`)
- CharGPT Example → Character Dataset: constructs dataset (evidence: `projects/chargpt/chargpt.py`)
- CharGPT Example → GPT Model: constructs model (evidence: `projects/chargpt/chargpt.py`)
- CharGPT Example → Training Loop: constructs trainer (evidence: `projects/chargpt/chargpt.py`)
- CharGPT Example → Config Utilities: uses utilities (evidence: `projects/chargpt/chargpt.py`)
- CharGPT Example → Text Sampling: registers callback (evidence: `projects/chargpt/chargpt.py`)
- Text Sampling → GPT Model: samples completion (evidence: `projects/chargpt/chargpt.py`)
- Developer → Sorting Demo: uses notebook (evidence: `README.md`)
- Sorting Demo → GPT Model: uses model and trainer (evidence: `README.md`)
- Developer → GPT-2 Generation: uses notebook (evidence: `README.md`)
- GPT-2 Generation → Transformers Hub: loads pretrained GPT-2 (evidence: `README.md`)
- GPT-2 Generation → GPT Model: generates text (evidence: `README.md`)
- GPT Model → Transformers Hub: imports pretrained weights (evidence: `mingpt/model.py`)
