# karpathy/nanochat architecture

> Architecture diagram of the GitHub repository karpathy/nanochat, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-10-26). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/karpathy/nanochat
- Repository: https://github.com/karpathy/nanochat

## Overview

nanochat is a compact experimental harness for training and using GPT language models on a single GPU node. The main workflow prepares text data and a tokenizer, trains a depth-configured GPT, saves checkpoints, then loads models for supervised or reinforcement fine-tuning, task evaluation, and interactive inference. The README also highlights compute-optimal model scaling and tokenizer evaluation. Edges cite README claims or SOURCE INDEX references; unsampled subsystem details are represented only where documented, so some relationships are omitted.

Main entry points: [`scripts/base_train.py`](https://github.com/karpathy/nanochat/blob/master/scripts/base_train.py) (Pretraining), [`nanochat/checkpoint_manager.py`](https://github.com/karpathy/nanochat/blob/master/nanochat/checkpoint_manager.py) (Checkpoint manager), [`tasks/common.py`](https://github.com/karpathy/nanochat/blob/master/tasks/common.py) (Task datasets), [`nanochat/engine.py`](https://github.com/karpathy/nanochat/blob/master/nanochat/engine.py) (Generation engine)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_data["Data and tokenization"]
  node_datafiles[("Pretraining corpus<br/>[dataset.py]")]
  node_loader["Training data loader<br/>[dataloader.py]"]
  node_toktrain["Tokenizer training<br/>[tok_train.py]"]
  node_tokenizer["Tokenizer<br/>[tokenizer.py]"]
  node_tok_eval["Tokenizer evaluation<br/>[tok_eval.py]"]
end

subgraph group_model["Model and training"]
  node_train["Pretraining<br/>[base_train.py]"]
  node_gpt["GPT transformer<br/>[gpt.py]"]
  node_attention["Attention kernels<br/>[flash_attention.py]"]
  node_optimizer["Optimizer<br/>[optim.py]"]
  node_fp8["FP8 training<br/>[fp8.py]"]
  node_engine["Generation engine<br/>[engine.py]"]
  node_checkpoints[("Checkpoint manager")]
end

subgraph group_chat["Chat adaptation"]
  node_sft["Supervised fine-tuning<br/>[chat_sft.py]"]
  node_rl["Reinforcement fine-tuning<br/>[chat_rl.py]"]
end

subgraph group_eval["Evaluation and inference"]
  node_taskdata["Task datasets<br/>[common.py]"]
  node_gsm["GSM8K<br/>[gsm8k.py]"]
  node_mmlu["MMLU<br/>[mmlu.py]"]
  node_othertasks["Other task suites"]
  node_baseeval["Pretraining evaluation<br/>[base_eval.py]"]
  node_coreeval["CORE metric<br/>[core_eval.py]"]
  node_chateval["Chat evaluation<br/>[chat_eval.py]"]
  node_cli["Chat CLI<br/>[chat_cli.py]"]
end

node_researcher(("Researcher"))
node_externaldata[("Dataset hub")]

node_researcher -->|"configures depth"| node_train
node_researcher -->|"trains tokenizer"| node_toktrain
node_researcher -->|"chats with model"| node_cli
node_datafiles -->|"provides documents"| node_loader
node_toktrain -->|"trains and saves"| node_tokenizer
node_tok_eval -->|"evaluates"| node_tokenizer
node_externaldata -.->|"loads datasets"| node_taskdata
node_gsm -->|"uses task base"| node_taskdata
node_mmlu -->|"uses task base"| node_taskdata
node_othertasks -.->|"uses task base"| node_taskdata
node_loader -->|"supplies batches"| node_train
node_train -->|"trains model"| node_gpt
node_train -.->|"updates weights"| node_optimizer
node_train -->|"enables FP8"| node_fp8
node_gpt -->|"calls attention"| node_attention
node_train -->|"saves state"| node_checkpoints
node_sft -->|"loads model"| node_checkpoints
node_sft -->|"uses conversations"| node_taskdata
node_sft -->|"evaluates samples"| node_engine
node_rl -->|"loads model"| node_checkpoints
node_rl -->|"uses math tasks"| node_gsm
node_baseeval -->|"loads model"| node_checkpoints
node_baseeval -->|"computes metric"| node_coreeval
node_baseeval -->|"evaluates model"| node_engine
node_chateval -->|"loads model"| node_checkpoints
node_chateval -->|"generates responses"| node_engine
node_chateval -->|"evaluates suites"| node_othertasks
node_cli -->|"loads model"| node_checkpoints
node_cli -->|"generates replies"| node_engine

click node_datafiles "https://github.com/karpathy/nanochat/blob/master/nanochat/dataset.py"
click node_loader "https://github.com/karpathy/nanochat/blob/master/nanochat/dataloader.py"
click node_toktrain "https://github.com/karpathy/nanochat/blob/master/scripts/tok_train.py"
click node_tokenizer "https://github.com/karpathy/nanochat/blob/master/nanochat/tokenizer.py"
click node_tok_eval "https://github.com/karpathy/nanochat/blob/master/scripts/tok_eval.py"
click node_train "https://github.com/karpathy/nanochat/blob/master/scripts/base_train.py"
click node_gpt "https://github.com/karpathy/nanochat/blob/master/nanochat/gpt.py"
click node_attention "https://github.com/karpathy/nanochat/blob/master/nanochat/flash_attention.py"
click node_optimizer "https://github.com/karpathy/nanochat/blob/master/nanochat/optim.py"
click node_fp8 "https://github.com/karpathy/nanochat/blob/master/nanochat/fp8.py"
click node_engine "https://github.com/karpathy/nanochat/blob/master/nanochat/engine.py"
click node_checkpoints "https://github.com/karpathy/nanochat/blob/master/nanochat/checkpoint_manager.py"
click node_sft "https://github.com/karpathy/nanochat/blob/master/scripts/chat_sft.py"
click node_rl "https://github.com/karpathy/nanochat/blob/master/scripts/chat_rl.py"
click node_taskdata "https://github.com/karpathy/nanochat/blob/master/tasks/common.py"
click node_gsm "https://github.com/karpathy/nanochat/blob/master/tasks/gsm8k.py"
click node_mmlu "https://github.com/karpathy/nanochat/blob/master/tasks/mmlu.py"
click node_othertasks "https://github.com/karpathy/nanochat/tree/master/tasks"
click node_baseeval "https://github.com/karpathy/nanochat/blob/master/scripts/base_eval.py"
click node_coreeval "https://github.com/karpathy/nanochat/blob/master/nanochat/core_eval.py"
click node_chateval "https://github.com/karpathy/nanochat/blob/master/scripts/chat_eval.py"
click node_cli "https://github.com/karpathy/nanochat/blob/master/scripts/chat_cli.py"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_datafiles,node_loader,node_toktrain,node_tokenizer,node_tok_eval toneBlue
class node_train,node_gpt,node_attention,node_optimizer,node_fp8,node_engine,node_checkpoints,node_externaldata toneAmber
class node_sft,node_rl toneMint
class node_taskdata,node_gsm,node_mmlu,node_othertasks,node_baseeval,node_coreeval,node_chateval,node_cli toneRose
class node_researcher toneIndigo
```

## Components

### Data and tokenization

- [Pretraining corpus](https://github.com/karpathy/nanochat/blob/master/nanochat/dataset.py): `nanochat/dataset.py`
- [Training data loader](https://github.com/karpathy/nanochat/blob/master/nanochat/dataloader.py): `nanochat/dataloader.py`
- [Tokenizer training](https://github.com/karpathy/nanochat/blob/master/scripts/tok_train.py): `scripts/tok_train.py`
- [Tokenizer](https://github.com/karpathy/nanochat/blob/master/nanochat/tokenizer.py): `nanochat/tokenizer.py`
- [Tokenizer evaluation](https://github.com/karpathy/nanochat/blob/master/scripts/tok_eval.py): `scripts/tok_eval.py`

### Model and training

- [Pretraining](https://github.com/karpathy/nanochat/blob/master/scripts/base_train.py): `scripts/base_train.py`
- [GPT transformer](https://github.com/karpathy/nanochat/blob/master/nanochat/gpt.py): `nanochat/gpt.py`
- [Attention kernels](https://github.com/karpathy/nanochat/blob/master/nanochat/flash_attention.py): `nanochat/flash_attention.py`
- [Optimizer](https://github.com/karpathy/nanochat/blob/master/nanochat/optim.py): `nanochat/optim.py`
- [FP8 training](https://github.com/karpathy/nanochat/blob/master/nanochat/fp8.py): `nanochat/fp8.py`
- [Generation engine](https://github.com/karpathy/nanochat/blob/master/nanochat/engine.py): `nanochat/engine.py`
- [Checkpoint manager](https://github.com/karpathy/nanochat/blob/master/nanochat/checkpoint_manager.py): `nanochat/checkpoint_manager.py`

### Chat adaptation

- [Supervised fine-tuning](https://github.com/karpathy/nanochat/blob/master/scripts/chat_sft.py): `scripts/chat_sft.py`
- [Reinforcement fine-tuning](https://github.com/karpathy/nanochat/blob/master/scripts/chat_rl.py): `scripts/chat_rl.py`

### Evaluation and inference

- [Task datasets](https://github.com/karpathy/nanochat/blob/master/tasks/common.py): `tasks/common.py`
- [GSM8K](https://github.com/karpathy/nanochat/blob/master/tasks/gsm8k.py): `tasks/gsm8k.py`
- [MMLU](https://github.com/karpathy/nanochat/blob/master/tasks/mmlu.py): `tasks/mmlu.py`
- [Other task suites](https://github.com/karpathy/nanochat/tree/master/tasks): `tasks`
- [Pretraining evaluation](https://github.com/karpathy/nanochat/blob/master/scripts/base_eval.py): `scripts/base_eval.py`
- [CORE metric](https://github.com/karpathy/nanochat/blob/master/nanochat/core_eval.py): `nanochat/core_eval.py`
- [Chat evaluation](https://github.com/karpathy/nanochat/blob/master/scripts/chat_eval.py): `scripts/chat_eval.py`
- [Chat CLI](https://github.com/karpathy/nanochat/blob/master/scripts/chat_cli.py): `scripts/chat_cli.py`

### Other components

- **Researcher**
- **Dataset hub**

## Connections

- Researcher → Pretraining: configures depth (evidence: `README.md`)
- Researcher → Tokenizer training: trains tokenizer
- Researcher → Chat CLI: chats with model (evidence: `README.md`)
- Pretraining corpus → Training data loader: provides documents (evidence: `nanochat/dataloader.py`)
- Tokenizer training → Tokenizer: trains and saves (evidence: `scripts/tok_train.py`)
- Tokenizer evaluation → Tokenizer: evaluates (evidence: `scripts/tok_eval.py`)
- Dataset hub → Task datasets: loads datasets
- GSM8K → Task datasets: uses task base (evidence: `tasks/gsm8k.py`)
- MMLU → Task datasets: uses task base (evidence: `tasks/mmlu.py`)
- Other task suites → Task datasets: uses task base (evidence: `tasks/gsm8k.py`)
- Training data loader → Pretraining: supplies batches (evidence: `scripts/base_train.py`)
- Pretraining → GPT transformer: trains model (evidence: `scripts/base_train.py`)
- Pretraining → Optimizer: updates weights
- Pretraining → FP8 training: enables FP8 (evidence: `scripts/base_train.py`)
- GPT transformer → Attention kernels: calls attention (evidence: `nanochat/gpt.py`)
- Pretraining → Checkpoint manager: saves state (evidence: `scripts/base_train.py`)
- Supervised fine-tuning → Checkpoint manager: loads model (evidence: `scripts/chat_sft.py`)
- Supervised fine-tuning → Task datasets: uses conversations (evidence: `scripts/chat_sft.py`)
- Supervised fine-tuning → Generation engine: evaluates samples (evidence: `scripts/chat_sft.py`)
- Reinforcement fine-tuning → Checkpoint manager: loads model (evidence: `scripts/chat_rl.py`)
- Reinforcement fine-tuning → GSM8K: uses math tasks (evidence: `scripts/chat_rl.py`)
- Pretraining evaluation → Checkpoint manager: loads model (evidence: `scripts/base_eval.py`)
- Pretraining evaluation → CORE metric: computes metric (evidence: `scripts/base_eval.py`)
- Pretraining evaluation → Generation engine: evaluates model (evidence: `scripts/base_eval.py`)
- Chat evaluation → Checkpoint manager: loads model (evidence: `scripts/chat_eval.py`)
- Chat evaluation → Generation engine: generates responses (evidence: `scripts/chat_eval.py`)
- Chat evaluation → Other task suites: evaluates suites (evidence: `scripts/chat_eval.py`)
- Chat CLI → Checkpoint manager: loads model (evidence: `scripts/chat_cli.py`)
- Chat CLI → Generation engine: generates replies (evidence: `scripts/chat_cli.py`)
