# white47/my-llm architecture

> Architecture diagram of the GitHub repository white47/my-llm, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-09-29). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/white47/my-llm
- Repository: https://github.com/white47/my-llm

## Overview

This repository is a hands-on LLM learning record, not one integrated application. Its documented route moves from GPT and tokenization experiments through pretraining, fine-tuning/alignment, and quantized chat deployment. The graph preserves those stages and the major alignment branches; where a capability is documented but its implementation was not sampled, paths use the existing enclosing directory and wiring is limited to documented behavior.

Main entry points: [`notes/Build-nanoGPT/train_gpt2.py`](https://github.com/white47/my-llm/blob/master/notes/Build-nanoGPT/train_gpt2.py) (Base Checkpoint), [`notes/Fine-tuning-and-Alignment`](https://github.com/white47/my-llm/tree/master/notes/Fine-tuning-and-Alignment) (Fine-tuned Weights), [`notes/Fine-tuning-and-Alignment/train_sft.py`](https://github.com/white47/my-llm/blob/master/notes/Fine-tuning-and-Alignment/train_sft.py) (SFT Training), [`notes/Fine-tuning-and-Alignment/train_lora.py`](https://github.com/white47/my-llm/blob/master/notes/Fine-tuning-and-Alignment/train_lora.py) (LoRA Fine-tuning)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_foundations["Foundations"]
  node_pytorch["PyTorch Basics"]
  node_gpt_from_scratch["GPT from Scratch<br/>[train_gpt2.py]"]
  node_bpe["BPE Tokenizer<br/>[tiktoken_gpt.py]"]
  node_tokenizer_examples["Tokenizer Examples"]
end

subgraph group_pretraining["Pretraining"]
  node_fineweb_download["FineWeb Shards<br/>[download_10BT.py]"]
  node_pretrain_loader["Pretraining Loader<br/>[train_gpt2.py]"]
  node_pretraining["GPT Pretraining<br/>[train_gpt2_r.py]"]
  node_base_checkpoint["Base Checkpoint<br/>[train_gpt2.py]"]
end

subgraph group_alignment["Fine-tuning &amp; Alignment"]
  node_sft["SFT Training<br/>[train_sft.py]"]
  node_lora["LoRA Fine-tuning<br/>[train_lora.py]"]
  node_reward_model["Reward Model"]
  node_dpo["DPO Alignment<br/>[train_dpo.py]"]
  node_ppo["PPO Alignment<br/>[train_ppo.py]"]
  node_aligned_checkpoint["Fine-tuned Weights"]
  node_inference["LoRA Inference<br/>[inference.py]"]
end

subgraph group_deployment["Deployment &amp; Chat"]
  node_gguf_conversion["GGUF Conversion<br/>[convert_to_gguf.py]"]
  node_llama_cpp["llama.cpp"]
  node_model_loader["Model Loader<br/>[model_loader.py]"]
  node_chat_app["Streamlit Chat<br/>[chat_app.py]"]
  node_chat_generation["Chat Generation<br/>[generate.py]"]
end

node_learner(("Learner"))

node_learner -->|"studies"| node_pytorch
node_learner -->|"runs experiments"| node_gpt_from_scratch
node_learner -->|"trains tokenizer"| node_bpe
node_learner -->|"explores"| node_tokenizer_examples
node_bpe -->|"demonstrates BPE"| node_tokenizer_examples
node_learner -->|"prepares data"| node_fineweb_download
node_fineweb_download -->|"writes token shards"| node_pretrain_loader
node_pretrain_loader -->|"supplies batches"| node_pretraining
node_pretraining -->|"saves weights"| node_base_checkpoint
node_learner -->|"fine-tunes"| node_sft
node_learner -->|"fine-tunes"| node_lora
node_base_checkpoint -->|"initializes"| node_sft
node_base_checkpoint -->|"initializes"| node_lora
node_base_checkpoint -->|"initializes"| node_reward_model
node_base_checkpoint -->|"initializes"| node_dpo
node_base_checkpoint -->|"initializes"| node_ppo
node_reward_model -->|"scores responses"| node_ppo
node_sft -->|"produces weights"| node_aligned_checkpoint
node_lora -->|"produces adapter"| node_aligned_checkpoint
node_dpo -->|"produces weights"| node_aligned_checkpoint
node_ppo -->|"produces weights"| node_aligned_checkpoint
node_base_checkpoint -->|"loads base"| node_inference
node_aligned_checkpoint -->|"loads adapter or checkpoint"| node_inference
node_aligned_checkpoint -->|"converts model"| node_gguf_conversion
node_gguf_conversion -.->|"exports for inference"| node_llama_cpp
node_chat_app -->|"loads model"| node_model_loader
node_chat_app -->|"requests generation"| node_chat_generation
node_model_loader -->|"provides model"| node_chat_generation

click node_pytorch "https://github.com/white47/my-llm/tree/master/notes/PyTorch"
click node_gpt_from_scratch "https://github.com/white47/my-llm/blob/master/notes/Build-nanoGPT/train_gpt2.py"
click node_bpe "https://github.com/white47/my-llm/blob/master/notes/GPT-Tokenizer/tiktoken_gpt.py"
click node_tokenizer_examples "https://github.com/white47/my-llm/tree/master/notes/GPT-Tokenizer"
click node_fineweb_download "https://github.com/white47/my-llm/blob/master/notes/Build-nanoGPT/download_10BT.py"
click node_pretrain_loader "https://github.com/white47/my-llm/blob/master/notes/Build-nanoGPT/train_gpt2.py"
click node_pretraining "https://github.com/white47/my-llm/blob/master/notes/Build-nanoGPT/train_gpt2_r.py"
click node_base_checkpoint "https://github.com/white47/my-llm/blob/master/notes/Build-nanoGPT/train_gpt2.py"
click node_sft "https://github.com/white47/my-llm/blob/master/notes/Fine-tuning-and-Alignment/train_sft.py"
click node_lora "https://github.com/white47/my-llm/blob/master/notes/Fine-tuning-and-Alignment/train_lora.py"
click node_reward_model "https://github.com/white47/my-llm/blob/master/notes/Fine-tuning-and-Alignment/train_reward_model.py"
click node_dpo "https://github.com/white47/my-llm/blob/master/notes/Fine-tuning-and-Alignment/train_dpo.py"
click node_ppo "https://github.com/white47/my-llm/blob/master/notes/Fine-tuning-and-Alignment/train_ppo.py"
click node_aligned_checkpoint "https://github.com/white47/my-llm/tree/master/notes/Fine-tuning-and-Alignment"
click node_inference "https://github.com/white47/my-llm/blob/master/notes/Fine-tuning-and-Alignment/inference.py"
click node_gguf_conversion "https://github.com/white47/my-llm/blob/master/notes/Quantification-and-Deployment/convert_to_gguf.py"
click node_model_loader "https://github.com/white47/my-llm/blob/master/notes/Quantification-and-Deployment/chat/model_loader.py"
click node_chat_app "https://github.com/white47/my-llm/blob/master/notes/Quantification-and-Deployment/chat/chat_app.py"
click node_chat_generation "https://github.com/white47/my-llm/blob/master/notes/Quantification-and-Deployment/chat/generate.py"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_pytorch,node_gpt_from_scratch,node_bpe,node_tokenizer_examples toneBlue
class node_fineweb_download,node_pretrain_loader,node_pretraining,node_base_checkpoint toneAmber
class node_sft,node_lora,node_reward_model,node_dpo,node_ppo,node_aligned_checkpoint,node_inference toneMint
class node_gguf_conversion,node_llama_cpp,node_model_loader,node_chat_app,node_chat_generation toneRose
class node_learner toneIndigo
```

## Components

### Foundations

- [PyTorch Basics](https://github.com/white47/my-llm/tree/master/notes/PyTorch): `notes/PyTorch`
- [GPT from Scratch](https://github.com/white47/my-llm/blob/master/notes/Build-nanoGPT/train_gpt2.py): `notes/Build-nanoGPT/train_gpt2.py`
- [BPE Tokenizer](https://github.com/white47/my-llm/blob/master/notes/GPT-Tokenizer/tiktoken_gpt.py): `notes/GPT-Tokenizer/tiktoken_gpt.py`
- [Tokenizer Examples](https://github.com/white47/my-llm/tree/master/notes/GPT-Tokenizer): `notes/GPT-Tokenizer`

### Pretraining

- [FineWeb Shards](https://github.com/white47/my-llm/blob/master/notes/Build-nanoGPT/download_10BT.py): `notes/Build-nanoGPT/download_10BT.py`
- [Pretraining Loader](https://github.com/white47/my-llm/blob/master/notes/Build-nanoGPT/train_gpt2.py): `notes/Build-nanoGPT/train_gpt2.py`
- [GPT Pretraining](https://github.com/white47/my-llm/blob/master/notes/Build-nanoGPT/train_gpt2_r.py): `notes/Build-nanoGPT/train_gpt2_r.py`
- [Base Checkpoint](https://github.com/white47/my-llm/blob/master/notes/Build-nanoGPT/train_gpt2.py): `notes/Build-nanoGPT/train_gpt2.py`

### Fine-tuning & Alignment

- [SFT Training](https://github.com/white47/my-llm/blob/master/notes/Fine-tuning-and-Alignment/train_sft.py): `notes/Fine-tuning-and-Alignment/train_sft.py`
- [LoRA Fine-tuning](https://github.com/white47/my-llm/blob/master/notes/Fine-tuning-and-Alignment/train_lora.py): `notes/Fine-tuning-and-Alignment/train_lora.py`
- [Reward Model](https://github.com/white47/my-llm/blob/master/notes/Fine-tuning-and-Alignment/train_reward_model.py): `notes/Fine-tuning-and-Alignment/train_reward_model.py`
- [DPO Alignment](https://github.com/white47/my-llm/blob/master/notes/Fine-tuning-and-Alignment/train_dpo.py): `notes/Fine-tuning-and-Alignment/train_dpo.py`
- [PPO Alignment](https://github.com/white47/my-llm/blob/master/notes/Fine-tuning-and-Alignment/train_ppo.py): `notes/Fine-tuning-and-Alignment/train_ppo.py`
- [Fine-tuned Weights](https://github.com/white47/my-llm/tree/master/notes/Fine-tuning-and-Alignment): `notes/Fine-tuning-and-Alignment`
- [LoRA Inference](https://github.com/white47/my-llm/blob/master/notes/Fine-tuning-and-Alignment/inference.py): `notes/Fine-tuning-and-Alignment/inference.py`

### Deployment & Chat

- [GGUF Conversion](https://github.com/white47/my-llm/blob/master/notes/Quantification-and-Deployment/convert_to_gguf.py): `notes/Quantification-and-Deployment/convert_to_gguf.py`
- **llama.cpp**
- [Model Loader](https://github.com/white47/my-llm/blob/master/notes/Quantification-and-Deployment/chat/model_loader.py): `notes/Quantification-and-Deployment/chat/model_loader.py`
- [Streamlit Chat](https://github.com/white47/my-llm/blob/master/notes/Quantification-and-Deployment/chat/chat_app.py): `notes/Quantification-and-Deployment/chat/chat_app.py`
- [Chat Generation](https://github.com/white47/my-llm/blob/master/notes/Quantification-and-Deployment/chat/generate.py): `notes/Quantification-and-Deployment/chat/generate.py`

### Other components

- **Learner**

## Connections

- Learner → PyTorch Basics: studies
- Learner → GPT from Scratch: runs experiments
- Learner → BPE Tokenizer: trains tokenizer
- Learner → Tokenizer Examples: explores
- BPE Tokenizer → Tokenizer Examples: demonstrates BPE
- Learner → FineWeb Shards: prepares data
- FineWeb Shards → Pretraining Loader: writes token shards
- Pretraining Loader → GPT Pretraining: supplies batches
- GPT Pretraining → Base Checkpoint: saves weights
- Learner → SFT Training: fine-tunes
- Learner → LoRA Fine-tuning: fine-tunes
- Base Checkpoint → SFT Training: initializes
- Base Checkpoint → LoRA Fine-tuning: initializes
- Base Checkpoint → Reward Model: initializes
- Base Checkpoint → DPO Alignment: initializes
- Base Checkpoint → PPO Alignment: initializes
- Reward Model → PPO Alignment: scores responses
- SFT Training → Fine-tuned Weights: produces weights
- LoRA Fine-tuning → Fine-tuned Weights: produces adapter
- DPO Alignment → Fine-tuned Weights: produces weights
- PPO Alignment → Fine-tuned Weights: produces weights
- Base Checkpoint → LoRA Inference: loads base
- Fine-tuned Weights → LoRA Inference: loads adapter or checkpoint
- Fine-tuned Weights → GGUF Conversion: converts model
- GGUF Conversion → llama.cpp: exports for inference
- Streamlit Chat → Model Loader: loads model
- Streamlit Chat → Chat Generation: requests generation
- Model Loader → Chat Generation: provides model