How karpathy/nn-zero-to-hero works
This repository is a course archive, not a reusable neural-network runtime: its product is video-led instruction accompanied by Jupyter notebooks. The learner follows micrograd’s backpropagation introduction and a sequence of makemore language-modeling notebooks, progressing from bigrams through MLPs, BatchNorm, manual backpropagation, and a WaveNet-like CNN. The README also documents GPT and tokenizer lectures, but this tree contains no corresponding notebooks. Relationships are based on the README; notebook contents were not available for inspection.
Main entry points: README.md Course guide · makemore_part2_mlp.ipynb MLP model · makemore_part1_bigrams.ipynb Bigram model · makemore_part3_bn.ipynb Activations and BatchNorm
Components 12
Course overview
- Course guide
README.md - Video lectures
- GPT lecture
README.md - Tokenizer lecture
README.md
Micrograd lecture
- Backprop notebooks
lectures/micrograd/micrograd_lecture_first_half_roughly.ipynb - Backprop continuation
lectures/micrograd/micrograd_lecture_second_half_roughly.ipynb
Makemore lectures
- Bigram model
lectures/makemore/makemore_part1_bigrams.ipynb - MLP model
lectures/makemore/makemore_part2_mlp.ipynb - Activations and BatchNorm
lectures/makemore/makemore_part3_bn.ipynb - Manual backprop
lectures/makemore/makemore_part4_backprop.ipynb - WaveNet-like CNN
lectures/makemore/makemore_part5_cnn1.ipynb
Other components
- Learner
Connections 17
- Learner → Video lectures: watches
README.md - Video lectures → Backprop notebooks: builds alongside
README.md - Video lectures → Bigram model: builds alongside
README.md - Learner → Course guide: uses guide
README.md - Course guide → Backprop notebooks: identifies notebook
README.md - Course guide → Backprop continuation: identifies notebook
README.md - Course guide → Bigram model: identifies notebook
README.md - Course guide → MLP model: identifies notebook
README.md - Course guide → Activations and BatchNorm: identifies notebook
README.md - Course guide → Manual backprop: identifies notebook
README.md - Course guide → WaveNet-like CNN: identifies notebook
README.md - Bigram model → MLP model: complexifies into
README.md - MLP model → Activations and BatchNorm: extends with
README.md - Activations and BatchNorm → Manual backprop: backpropagates through
README.md - MLP model → WaveNet-like CNN: deepens into
README.md - Course guide → GPT lecture: documents
README.md - Course guide → Tokenizer lecture: documents
README.md