# karpathy/nn-zero-to-hero architecture

> Architecture diagram of the GitHub repository karpathy/nn-zero-to-hero, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-11-09). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/karpathy/nn-zero-to-hero
- Repository: https://github.com/karpathy/nn-zero-to-hero

## Overview

This repository is a course archive, not a reusable neural-network runtime: its product is video-led instruction accompanied by Jupyter notebooks. The learner follows micrograd’s backpropagation introduction and a sequence of makemore language-modeling notebooks, progressing from bigrams through MLPs, BatchNorm, manual backpropagation, and a WaveNet-like CNN. The README also documents GPT and tokenizer lectures, but this tree contains no corresponding notebooks. Relationships are based on the README; notebook contents were not available for inspection.

Main entry points: [`README.md`](https://github.com/karpathy/nn-zero-to-hero/blob/master/README.md) (Course guide), [`lectures/makemore/makemore_part2_mlp.ipynb`](https://github.com/karpathy/nn-zero-to-hero/blob/master/lectures/makemore/makemore_part2_mlp.ipynb) (MLP model), [`lectures/makemore/makemore_part1_bigrams.ipynb`](https://github.com/karpathy/nn-zero-to-hero/blob/master/lectures/makemore/makemore_part1_bigrams.ipynb) (Bigram model), [`lectures/makemore/makemore_part3_bn.ipynb`](https://github.com/karpathy/nn-zero-to-hero/blob/master/lectures/makemore/makemore_part3_bn.ipynb) (Activations and BatchNorm)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_course["Course overview"]
  node_course_readme["Course guide<br/>[README.md]"]
  node_video_lectures(("Video lectures"))
  node_gpt_lecture["GPT lecture<br/>[README.md]"]
  node_tokenizer_lecture["Tokenizer lecture<br/>[README.md]"]
end

subgraph group_micrograd["Micrograd lecture"]
  node_micrograd_notebooks["Backprop notebooks"]
  node_micrograd_second["Backprop continuation"]
end

subgraph group_makemore["Makemore lectures"]
  node_bigrams["Bigram model"]
  node_mlp["MLP model"]
  node_batchnorm["Activations and BatchNorm"]
  node_manual_backprop["Manual backprop"]
  node_wavenet["WaveNet-like CNN"]
end

node_learner(("Learner"))

node_learner -->|"watches"| node_video_lectures
node_video_lectures -->|"builds alongside"| node_micrograd_notebooks
node_video_lectures -->|"builds alongside"| node_bigrams
node_learner -->|"uses guide"| node_course_readme
node_course_readme -->|"identifies notebook"| node_micrograd_notebooks
node_course_readme -->|"identifies notebook"| node_micrograd_second
node_course_readme -->|"identifies notebook"| node_bigrams
node_course_readme -->|"identifies notebook"| node_mlp
node_course_readme -->|"identifies notebook"| node_batchnorm
node_course_readme -->|"identifies notebook"| node_manual_backprop
node_course_readme -->|"identifies notebook"| node_wavenet
node_bigrams -->|"complexifies into"| node_mlp
node_mlp -->|"extends with"| node_batchnorm
node_batchnorm -->|"backpropagates through"| node_manual_backprop
node_mlp -->|"deepens into"| node_wavenet
node_course_readme -->|"documents"| node_gpt_lecture
node_course_readme -->|"documents"| node_tokenizer_lecture

click node_course_readme "https://github.com/karpathy/nn-zero-to-hero/blob/master/README.md"
click node_gpt_lecture "https://github.com/karpathy/nn-zero-to-hero/blob/master/README.md"
click node_tokenizer_lecture "https://github.com/karpathy/nn-zero-to-hero/blob/master/README.md"
click node_micrograd_notebooks "https://github.com/karpathy/nn-zero-to-hero/blob/master/lectures/micrograd/micrograd_lecture_first_half_roughly.ipynb"
click node_micrograd_second "https://github.com/karpathy/nn-zero-to-hero/blob/master/lectures/micrograd/micrograd_lecture_second_half_roughly.ipynb"
click node_bigrams "https://github.com/karpathy/nn-zero-to-hero/blob/master/lectures/makemore/makemore_part1_bigrams.ipynb"
click node_mlp "https://github.com/karpathy/nn-zero-to-hero/blob/master/lectures/makemore/makemore_part2_mlp.ipynb"
click node_batchnorm "https://github.com/karpathy/nn-zero-to-hero/blob/master/lectures/makemore/makemore_part3_bn.ipynb"
click node_manual_backprop "https://github.com/karpathy/nn-zero-to-hero/blob/master/lectures/makemore/makemore_part4_backprop.ipynb"
click node_wavenet "https://github.com/karpathy/nn-zero-to-hero/blob/master/lectures/makemore/makemore_part5_cnn1.ipynb"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_course_readme,node_video_lectures,node_gpt_lecture,node_tokenizer_lecture toneBlue
class node_micrograd_notebooks,node_micrograd_second toneAmber
class node_bigrams,node_mlp,node_batchnorm,node_manual_backprop,node_wavenet toneMint
class node_learner toneIndigo
```

## Components

### Course overview

- [Course guide](https://github.com/karpathy/nn-zero-to-hero/blob/master/README.md): `README.md`
- **Video lectures**
- [GPT lecture](https://github.com/karpathy/nn-zero-to-hero/blob/master/README.md): `README.md`
- [Tokenizer lecture](https://github.com/karpathy/nn-zero-to-hero/blob/master/README.md): `README.md`

### Micrograd lecture

- [Backprop notebooks](https://github.com/karpathy/nn-zero-to-hero/blob/master/lectures/micrograd/micrograd_lecture_first_half_roughly.ipynb): `lectures/micrograd/micrograd_lecture_first_half_roughly.ipynb`
- [Backprop continuation](https://github.com/karpathy/nn-zero-to-hero/blob/master/lectures/micrograd/micrograd_lecture_second_half_roughly.ipynb): `lectures/micrograd/micrograd_lecture_second_half_roughly.ipynb`

### Makemore lectures

- [Bigram model](https://github.com/karpathy/nn-zero-to-hero/blob/master/lectures/makemore/makemore_part1_bigrams.ipynb): `lectures/makemore/makemore_part1_bigrams.ipynb`
- [MLP model](https://github.com/karpathy/nn-zero-to-hero/blob/master/lectures/makemore/makemore_part2_mlp.ipynb): `lectures/makemore/makemore_part2_mlp.ipynb`
- [Activations and BatchNorm](https://github.com/karpathy/nn-zero-to-hero/blob/master/lectures/makemore/makemore_part3_bn.ipynb): `lectures/makemore/makemore_part3_bn.ipynb`
- [Manual backprop](https://github.com/karpathy/nn-zero-to-hero/blob/master/lectures/makemore/makemore_part4_backprop.ipynb): `lectures/makemore/makemore_part4_backprop.ipynb`
- [WaveNet-like CNN](https://github.com/karpathy/nn-zero-to-hero/blob/master/lectures/makemore/makemore_part5_cnn1.ipynb): `lectures/makemore/makemore_part5_cnn1.ipynb`

### Other components

- **Learner**

## Connections

- Learner → Video lectures: watches (evidence: `README.md`)
- Video lectures → Backprop notebooks: builds alongside (evidence: `README.md`)
- Video lectures → Bigram model: builds alongside (evidence: `README.md`)
- Learner → Course guide: uses guide (evidence: `README.md`)
- Course guide → Backprop notebooks: identifies notebook (evidence: `README.md`)
- Course guide → Backprop continuation: identifies notebook (evidence: `README.md`)
- Course guide → Bigram model: identifies notebook (evidence: `README.md`)
- Course guide → MLP model: identifies notebook (evidence: `README.md`)
- Course guide → Activations and BatchNorm: identifies notebook (evidence: `README.md`)
- Course guide → Manual backprop: identifies notebook (evidence: `README.md`)
- Course guide → WaveNet-like CNN: identifies notebook (evidence: `README.md`)
- Bigram model → MLP model: complexifies into (evidence: `README.md`)
- MLP model → Activations and BatchNorm: extends with (evidence: `README.md`)
- Activations and BatchNorm → Manual backprop: backpropagates through (evidence: `README.md`)
- MLP model → WaveNet-like CNN: deepens into (evidence: `README.md`)
- Course guide → GPT lecture: documents (evidence: `README.md`)
- Course guide → Tokenizer lecture: documents (evidence: `README.md`)
