# karpathy/llm101n architecture

> Architecture diagram of the GitHub repository karpathy/llm101n, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-11-09). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/karpathy/llm101n
- Repository: https://github.com/karpathy/llm101n

## Overview

This repository documents a planned educational course, not an implemented software product; the README says the course is still being developed and the repository is archived. Its principal workflow is curricular: learners progress from language-model and neural-network fundamentals through training and inference to deployment, with a planned Storyteller web app as the outcome. The graph preserves the documented syllabus stages and major capabilities; all relationships represent course progression or stated goals, not inspected runtime calls.

Main entry points: [`README.md`](https://github.com/karpathy/llm101n/blob/master/README.md) (Bigram Model)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_foundations["Model Foundations"]
  node_bigram["Bigram Model<br/>[README.md]"]
  node_micrograd["Micrograd<br/>[README.md]"]
  node_ngram["N-gram Model<br/>[README.md]"]
end

subgraph group_architecture["LLM Architecture"]
  node_attention["Attention<br/>[README.md]"]
  node_transformer["Transformer<br/>[README.md]"]
  node_tokenization["Tokenization<br/>[README.md]"]
end

subgraph group_training_inference["Training and Inference"]
  node_optimization["Optimization<br/>[README.md]"]
  node_performance["Performance<br/>[README.md]"]
  node_datasets["Datasets<br/>[README.md]"]
  node_inference["Inference<br/>[README.md]"]
  node_sft["SFT Fine-tuning<br/>[README.md]"]
  node_rl["RL Fine-tuning<br/>[README.md]"]
end

subgraph group_product_extensions["Product and Extensions"]
  node_deployment["Deployment<br/>[README.md]"]
  node_storyteller["Storyteller App<br/>[README.md]"]
  node_multimodal["Multimodal<br/>[README.md]"]
end

node_learner(("Learner"))

node_learner -->|"starts course"| node_bigram
node_bigram -->|"then studies"| node_micrograd
node_micrograd -->|"then studies"| node_ngram
node_ngram -->|"then studies"| node_attention
node_attention -->|"then studies"| node_transformer
node_transformer -->|"then studies"| node_tokenization
node_tokenization -->|"then studies"| node_optimization
node_optimization -->|"then studies"| node_performance
node_performance -->|"then studies"| node_datasets
node_datasets -->|"then studies"| node_inference
node_inference -->|"then studies"| node_sft
node_sft -->|"then studies"| node_rl
node_rl -->|"then studies"| node_deployment
node_deployment -->|"targets app"| node_storyteller
node_storyteller -->|"course continues"| node_multimodal

click node_bigram "https://github.com/karpathy/llm101n/blob/master/README.md"
click node_micrograd "https://github.com/karpathy/llm101n/blob/master/README.md"
click node_ngram "https://github.com/karpathy/llm101n/blob/master/README.md"
click node_attention "https://github.com/karpathy/llm101n/blob/master/README.md"
click node_transformer "https://github.com/karpathy/llm101n/blob/master/README.md"
click node_tokenization "https://github.com/karpathy/llm101n/blob/master/README.md"
click node_optimization "https://github.com/karpathy/llm101n/blob/master/README.md"
click node_performance "https://github.com/karpathy/llm101n/blob/master/README.md"
click node_datasets "https://github.com/karpathy/llm101n/blob/master/README.md"
click node_inference "https://github.com/karpathy/llm101n/blob/master/README.md"
click node_sft "https://github.com/karpathy/llm101n/blob/master/README.md"
click node_rl "https://github.com/karpathy/llm101n/blob/master/README.md"
click node_deployment "https://github.com/karpathy/llm101n/blob/master/README.md"
click node_storyteller "https://github.com/karpathy/llm101n/blob/master/README.md"
click node_multimodal "https://github.com/karpathy/llm101n/blob/master/README.md"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_bigram,node_micrograd,node_ngram toneBlue
class node_attention,node_transformer,node_tokenization toneAmber
class node_optimization,node_performance,node_datasets,node_inference,node_sft,node_rl toneMint
class node_deployment,node_storyteller,node_multimodal toneRose
class node_learner toneIndigo
```

## Components

### Model Foundations

- [Bigram Model](https://github.com/karpathy/llm101n/blob/master/README.md): `README.md`
- [Micrograd](https://github.com/karpathy/llm101n/blob/master/README.md): `README.md`
- [N-gram Model](https://github.com/karpathy/llm101n/blob/master/README.md): `README.md`

### LLM Architecture

- [Attention](https://github.com/karpathy/llm101n/blob/master/README.md): `README.md`
- [Transformer](https://github.com/karpathy/llm101n/blob/master/README.md): `README.md`
- [Tokenization](https://github.com/karpathy/llm101n/blob/master/README.md): `README.md`

### Training and Inference

- [Optimization](https://github.com/karpathy/llm101n/blob/master/README.md): `README.md`
- [Performance](https://github.com/karpathy/llm101n/blob/master/README.md): `README.md`
- [Datasets](https://github.com/karpathy/llm101n/blob/master/README.md): `README.md`
- [Inference](https://github.com/karpathy/llm101n/blob/master/README.md): `README.md`
- [SFT Fine-tuning](https://github.com/karpathy/llm101n/blob/master/README.md): `README.md`
- [RL Fine-tuning](https://github.com/karpathy/llm101n/blob/master/README.md): `README.md`

### Product and Extensions

- [Deployment](https://github.com/karpathy/llm101n/blob/master/README.md): `README.md`
- [Storyteller App](https://github.com/karpathy/llm101n/blob/master/README.md): `README.md`
- [Multimodal](https://github.com/karpathy/llm101n/blob/master/README.md): `README.md`

### Other components

- **Learner**

## Connections

- Learner → Bigram Model: starts course (evidence: `README.md`)
- Bigram Model → Micrograd: then studies (evidence: `README.md`)
- Micrograd → N-gram Model: then studies (evidence: `README.md`)
- N-gram Model → Attention: then studies (evidence: `README.md`)
- Attention → Transformer: then studies (evidence: `README.md`)
- Transformer → Tokenization: then studies (evidence: `README.md`)
- Tokenization → Optimization: then studies (evidence: `README.md`)
- Optimization → Performance: then studies (evidence: `README.md`)
- Performance → Datasets: then studies (evidence: `README.md`)
- Datasets → Inference: then studies (evidence: `README.md`)
- Inference → SFT Fine-tuning: then studies (evidence: `README.md`)
- SFT Fine-tuning → RL Fine-tuning: then studies (evidence: `README.md`)
- RL Fine-tuning → Deployment: then studies (evidence: `README.md`)
- Deployment → Storyteller App: targets app (evidence: `README.md`)
- Storyteller App → Multimodal: course continues (evidence: `README.md`)
