# aburkov/thelmbook architecture

> Architecture diagram of the GitHub repository aburkov/thelmbook, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-05-12). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/aburkov/thelmbook
- Repository: https://github.com/aburkov/thelmbook

## Overview

##### Project type and purpose
- This repo is not a traditional application or service—it’s an educational/book project (“The Hundred-Page Language Models Book”) made up of example code, Jupyter notebooks, and Markdown documentation. Its purpose is to teach language-model concepts through runnable experiments and narrative wiki pages.

##### File structure
- Top level:
- A set of .py scripts (e.g. GRPO.py, quadratic_loss.py, embedding_vs_linear.py) that implement core experiments or utilities.
- A set of Jupyter notebooks (.ipynb) demonstrating specific language-model workflows (e.g. sampling_method.ipynb, emotion_GPT2_as_classifier.ipynb).
- README.md with book title and cover image.
- “wiki/”: directory of Markdown pages forming the narrative text of the book (each topic from tokenization to distributed training).
- No dedicated build scripts or static-site generator config—this repo is structured as source artifacts (notebooks + docs) served directly by GitHub.

##### README
- Confirms this is the “Hundred-Page Language Models Book” and shows the cover. No further architectural details.

## Diagram (Mermaid)

```mermaid
flowchart TD
    %% Infrastructure Layer
    GH["GitHub Repository"]:::infra
    README["README.md"]:::infra

    %% Code Experiments Layer
    subgraph "Code Experiments (Python Scripts)"
        direction TB
        GRPO["GRPO.py"]:::code
        embedding["embedding_vs_linear.py"]:::code
        quad["quadratic_loss.py"]:::code
        spotify["spotify_gemini_playlist.py"]:::code
    end

    %% Notebook Layer
    subgraph "Notebook Layer (Jupyter Notebooks)"
        direction TB
        nb1["GRPO_From_Scratch_Multi_GPU_DataParallel_Qwen_2_5_1_5B_Instruct.ipynb"]:::notebook
        nb2["GRPO_Qwen_0_5_Instruct.ipynb"]:::notebook
        nb3["byte_pair_encoding.ipynb"]:::notebook
        nb4["count_language_model.ipynb"]:::notebook
        nb5["emotion_GPT2_as_classifier.ipynb"]:::notebook
        nb6["emotion_GPT2_as_text_generator.ipynb"]:::notebook
        nb7["emotion_GPT2_as_text_generator_LoRA.ipynb"]:::notebook
        nb8["emotion_classifier_CNN.ipynb"]:::notebook
        nb9["emotion_classifier_LR.ipynb"]:::notebook
        nb10["instruct_GPT2.ipynb"]:::notebook
        nb11["news_RNN_language_model.ipynb"]:::notebook
        nb12["news_decoder_language_model.ipynb"]:::notebook
        nb13["sampling_method.ipynb"]:::notebook
    end

    %% Documentation Layer
    subgraph "Wiki Documentation (Markdown)"
        direction TB
        w1["wiki/index.md"]:::docs
        w2["wiki/tokenization.md"]:::docs
        w3["wiki/embeddings.md"]:::docs
        w4["wiki/encoder.md"]:::docs
        w5["wiki/encoder-decoder.md"]:::docs
        w6["wiki/non-transformer.md"]:::docs
        w7["wiki/math.md"]:::docs
        w8["wiki/evaluation.md"]:::docs
        w9["wiki/prompting.md"]:::docs
        w10["wiki/inference.md"]:::docs
        w11["wiki/online-finetuning.md"]:::docs
        w12["wiki/scaling.md"]:::docs
        w13["wiki/overfitting.md"]:::docs
        w14["wiki/compression.md"]:::docs
        w15["wiki/merging.md"]:::docs
        w16["wiki/alignment.md"]:::docs
        w17["wiki/security.md"]:::docs
        w18["wiki/deployment.md"]:::docs
        w19["wiki/distributed.md"]:::docs
        w20["wiki/GPU-rental.md"]:::docs
        w21["wiki/colabs.md"]:::docs
        w22["wiki/notebook-services.md"]:::docs
        w23["wiki/function-calling.md"]:::docs
        w24["wiki/scripts.md"]:::docs
        w25["wiki/test.md"]:::docs
        w26["wiki/VLM.md"]:::docs
        w27["wiki/PyTorch.md"]:::docs
        w28["wiki/corrections.md"]:::docs
    end

    %% End-User Layer
    Reader["End-User / Reader"]:::infra

    %% Connections
    GH --> README
    GH --> GRPO
    GH --> embedding
    GH --> quad
    GH --> spotify
    GH --> nb1
    GH --> nb2
    GH --> nb3
    GH --> nb4
    GH --> nb5
    GH --> nb6
    GH --> nb7
    GH --> nb8
    GH --> nb9
    GH --> nb10
    GH --> nb11
    GH --> nb12
    GH --> nb13
    GH --> w1
    GH --> w2
    GH --> w3
    GH --> w4
    GH --> w5
    GH --> w6
    GH --> w7
    GH --> w8
    GH --> w9
    GH --> w10
    GH --> w11
    GH --> w12
    GH --> w13
    GH --> w14
    GH --> w15
    GH --> w16
    GH --> w17
    GH --> w18
    GH --> w19
    GH --> w20
    GH --> w21
    GH --> w22
    GH --> w23
    GH --> w24
    GH --> w25
    GH --> w26
    GH --> w27
    GH --> w28

    GRPO --> nb1
    embedding --> nb3
    quad --> nb4
    spotify --> nb13

    nb1 --> w24
    nb2 --> w24
    nb3 --> w2
    nb4 --> w7
    nb5 --> w17
    nb6 --> w9
    nb7 --> w9
    nb8 --> w17
    nb9 --> w17
    nb10 --> w23
    nb11 --> w19
    nb12 --> w19
    nb13 --> w10

    nb1 --> Reader
    nb2 --> Reader
    nb3 --> Reader
    nb4 --> Reader
    nb5 --> Reader
    nb6 --> Reader
    nb7 --> Reader
    nb8 --> Reader
    nb9 --> Reader
    nb10 --> Reader
    nb11 --> Reader
    nb12 --> Reader
    nb13 --> Reader

    w1 --> Reader
    w2 --> Reader
    w3 --> Reader
    w4 --> Reader
    w5 --> Reader
    w6 --> Reader
    w7 --> Reader
    w8 --> Reader
    w9 --> Reader
    w10 --> Reader
    w11 --> Reader
    w12 --> Reader
    w13 --> Reader
    w14 --> Reader
    w15 --> Reader
    w16 --> Reader
    w17 --> Reader
    w18 --> Reader
    w19 --> Reader
    w20 --> Reader
    w21 --> Reader
    w22 --> Reader
    w23 --> Reader
    w24 --> Reader
    w25 --> Reader
    w26 --> Reader
    w27 --> Reader
    w28 --> Reader

    %% Click Events
    click GRPO "https://github.com/aburkov/thelmbook/blob/main/GRPO.py"
    click embedding "https://github.com/aburkov/thelmbook/blob/main/embedding_vs_linear.py"
    click quad "https://github.com/aburkov/thelmbook/blob/main/quadratic_loss.py"
    click spotify "https://github.com/aburkov/thelmbook/blob/main/spotify_gemini_playlist.py"
    click nb1 "https://github.com/aburkov/thelmbook/blob/main/GRPO_From_Scratch_Multi_GPU_DataParallel_Qwen_2_5_1_5B_Instruct.ipynb"
    click nb2 "https://github.com/aburkov/thelmbook/blob/main/GRPO_Qwen_0_5_Instruct.ipynb"
    click nb3 "https://github.com/aburkov/thelmbook/blob/main/byte_pair_encoding.ipynb"
    click nb4 "https://github.com/aburkov/thelmbook/blob/main/count_language_model.ipynb"
    click nb5 "https://github.com/aburkov/thelmbook/blob/main/emotion_GPT2_as_classifier.ipynb"
    click nb6 "https://github.com/aburkov/thelmbook/blob/main/emotion_GPT2_as_text_generator.ipynb"
    click nb7 "https://github.com/aburkov/thelmbook/blob/main/emotion_GPT2_as_text_generator_LoRA.ipynb"
    click nb8 "https://github.com/aburkov/thelmbook/blob/main/emotion_classifier_CNN.ipynb"
    click nb9 "https://github.com/aburkov/thelmbook/blob/main/emotion_classifier_LR.ipynb"
    click nb10 "https://github.com/aburkov/thelmbook/blob/main/instruct_GPT2.ipynb"
    click nb11 "https://github.com/aburkov/thelmbook/blob/main/news_RNN_language_model.ipynb"
    click nb12 "https://github.com/aburkov/thelmbook/blob/main/news_decoder_language_model.ipynb"
    click nb13 "https://github.com/aburkov/thelmbook/blob/main/sampling_method.ipynb"
    click w1 "https://github.com/aburkov/thelmbook/blob/main/wiki/index.md"
    click w2 "https://github.com/aburkov/thelmbook/blob/main/wiki/tokenization.md"
    click w3 "https://github.com/aburkov/thelmbook/blob/main/wiki/embeddings.md"
    click w4 "https://github.com/aburkov/thelmbook/blob/main/wiki/encoder.md"
    click w5 "https://github.com/aburkov/thelmbook/blob/main/wiki/encoder-decoder.md"
    click w6 "https://github.com/aburkov/thelmbook/blob/main/wiki/non-transformer.md"
    click w7 "https://github.com/aburkov/thelmbook/blob/main/wiki/math.md"
    click w8 "https://github.com/aburkov/thelmbook/blob/main/wiki/evaluation.md"
    click w9 "https://github.com/aburkov/thelmbook/blob/main/wiki/prompting.md"
    click w10 "https://github.com/aburkov/thelmbook/blob/main/wiki/inference.md"
    click w11 "https://github.com/aburkov/thelmbook/blob/main/wiki/online-finetuning.md"
    click w12 "https://github.com/aburkov/thelmbook/blob/main/wiki/scaling.md"
    click w13 "https://github.com/aburkov/thelmbook/blob/main/wiki/overfitting.md"
    click w14 "https://github.com/aburkov/thelmbook/blob/main/wiki/compression.md"
    click w15 "https://github.com/aburkov/thelmbook/blob/main/wiki/merging.md"
    click w16 "https://github.com/aburkov/thelmbook/blob/main/wiki/alignment.md"
    click w17 "https://github.com/aburkov/thelmbook/blob/main/wiki/security.md"
    click w18 "https://github.com/aburkov/thelmbook/blob/main/wiki/deployment.md"
    click w19 "https://github.com/aburkov/thelmbook/blob/main/wiki/distributed.md"
    click w20 "https://github.com/aburkov/thelmbook/blob/main/wiki/GPU-rental.md"
    click w21 "https://github.com/aburkov/thelmbook/blob/main/wiki/colabs.md"
    click w22 "https://github.com/aburkov/thelmbook/blob/main/wiki/notebook-services.md"
    click w23 "https://github.com/aburkov/thelmbook/blob/main/wiki/function-calling.md"
    click w24 "https://github.com/aburkov/thelmbook/blob/main/wiki/scripts.md"
    click w25 "https://github.com/aburkov/thelmbook/blob/main/wiki/test.md"
    click w26 "https://github.com/aburkov/thelmbook/blob/main/wiki/VLM.md"
    click w27 "https://github.com/aburkov/thelmbook/blob/main/wiki/PyTorch.md"
    click w28 "https://github.com/aburkov/thelmbook/blob/main/wiki/corrections.md"
    click README "https://github.com/aburkov/thelmbook/blob/main/README.md"

    %% Styles
    classDef code fill:#AED6F1,stroke:#21618C,stroke-width:1px
    classDef notebook fill:#F5B041,stroke:#B9770E,stroke-width:1px
    classDef docs fill:#ABEBC6,stroke:#239B56,stroke-width:1px
    classDef infra fill:#D5D8DC,stroke:#566573,stroke-width:1px
```
