# jj-devhub/llms-from-scratch architecture

> Architecture diagram of the GitHub repository jj-devhub/llms-from-scratch, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-11-27). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/jj-devhub/llms-from-scratch
- Repository: https://github.com/jj-devhub/llms-from-scratch

## Overview

##### Project Type and Purpose
- This is an educational, open-source code repository accompanying the book “Build a Large Language Model (From Scratch).”
- It is not a web application but a monorepo containing:
- Chapter‐by‐chapter Jupyter notebooks and Python scripts demonstrating how to build, pretrain, and fine-tune a GPT-style model from first principles.
- A reusable Python package (`pkg/llms_from_scratch`) that encapsulates the core implementations (tokenization, model definition, KV cache, generation, finetuning).
- Setup files, CI/CD workflows, tests, and bonus materials.

##### File Structure Analysis
- Top-level directories:
- `ch01`–`ch07` + `appendix-*`: chapter code and exercises.
- `pkg/llms_from_scratch`: production-style Python package implementing the core library.
- `setup`: environment setup recommendations (Python, Docker, AWS).
- `.github/workflows`: CI pipelines (GitHub Actions for testing and linting).
- `requirements.txt` / `pyproject.toml` / `pixi.toml`: build and dependency configs.
- Patterns & architectural cues:
- Separation of pedagogy (notebooks + sample scripts) vs. library code (`pkg`).
- Scripts invoking `pkg` modules for training (`gpt_train.py`), generation (`gpt_generate.py`), finetuning, evaluation.
- CI orchestrates test runs across OSes and Python/PyTorch versions.
- Optional environment support (Docker, SageMaker, local Python).

##### README Insights
- Confirms the repo is code‐for‐book: no microservices, but a modular library + notebooks + CLI scripts.
- Mentions external integration points: PyTorch (core DL), Hugging Face weight loading, OpenAI & Ollama APIs (in evaluation chapters).
- Visual “mental model” outlines stages: data loading → model implementation → training → finetuning → evaluation.

##### Components & Interactions
a. Main Components
1. **Core Library (`pkg/llms_from_scratch`)**
– Submodules: tokenizers, model definitions (GPT variants, KV cache), utilities.
2. **Chapter Notebooks & Sample Scripts**
– Jupyter notebooks for each chapter.
– Python scripts for training, generation, finetuning, evaluation, UIs (`app.py`).
3. **Data Layer**
– Text datasets (e.g., Gutenberg, IMDb reviews).
– Pretrained weight files (loaded via HF or local).
4. **External Services**
– PyTorch backend (CPU/GPU).
– Hugging Face models & tokenizers.
– OpenAI / Ollama APIs for evaluation.
5. **CI/CD & Environment Setup**
– GitHub Actions workflows.
– Docker/SageMaker devcontainers.
– `requirements.txt` / `pyproject.toml`.

b. Relationships & Interactions
- **Notebooks/Scripts** import from **Core Library** and load **Data**.
- **Core Library** calls **PyTorch** for tensor ops and GPU execution.
- **Training Scripts** read data, build model definitions, run training loops, write checkpoints.
- **Generation/Evaluation Scripts** load checkpoints, invoke generation routines, call external APIs if instructed.
- **CI Workflows** install dependencies, run `pytest` on `pkg/tests` and sample scripts, enforce linting, and deploy (optional).

c. Architectural Patterns & Principles
- **Modular Library Pattern**: the `pkg` directory exposes a well-defined API used by all chapter code.
- **Layered Separation**:
1. Data ingestion layer
2. Model definition & training layer
3. Application layer (script/notebook UIs)
4. Infrastructure layer (CI, Docker, cloud).
- **Reproducibility & Automation**: via CI pipelines and environment setup docs.

d. Key Technologies & Frameworks
- PyTorch (core DL computations)
- Python packaging (pyproject.toml, Pixi, pkg/…)
- Jupyter Notebooks
- GitHub Actions
- Docker / Devcontainers
- External APIs (Hugging Face, OpenAI, Ollama)

##### Final Recommendations
- **all layers**: code, data, compute backend, external services, infrastructure.

## Diagram (Mermaid)

```mermaid
flowchart TB
    %% Data Layer
    subgraph "Data Layer"
        direction TB
        Data1["Text Datasets (Gutenberg, IMDb)"]:::data
        Data2["Pretrained Weights"]:::data
    end

    %% Core Library
    subgraph "Core Library (pkg/llms_from_scratch)" 
        direction TB
        Utils["utils.py"]:::library
        MD02["ch02.py"]:::library
        MD03["ch03.py"]:::library
        MD04["ch04.py"]:::library
        MD05["ch05.py"]:::library
        MD06["ch06.py"]:::library
        MD07["ch07.py"]:::library
        Generate["generate.py"]:::library
        subgraph "KV Cache Modules"
            direction TB
            KV1["kv_cache/"]:::library
            KV2["kv_cache_batched/"]:::library
        end
    end

    %% Chapter Notebooks & Scripts
    subgraph "Chapter Notebooks & Sample Scripts"
        direction TB
        CH01["ch01/"]:::notebooks
        CH02["ch02/01_main-chapter-code/"]:::notebooks
        CH03["ch03/01_main-chapter-code/"]:::notebooks
        CH04["ch04/01_main-chapter-code/"]:::notebooks
        CH05["ch05/01_main-chapter-code/"]:::notebooks
        CH06["ch06/01_main-chapter-code/"]:::notebooks
        CH07["ch07/01_main-chapter-code/"]:::notebooks
        APPA["appendix-A/01_main-chapter-code/"]:::notebooks
        APPD["appendix-D/01_main-chapter-code/"]:::notebooks
        APPE["appendix-E/01_main-chapter-code/"]:::notebooks
    end

    %% External Services
    subgraph "External Services"
        direction TB
        PT["PyTorch Backend"]:::external
        HF["Hugging Face API"]:::external
        OA["OpenAI API"]:::external
        OL["Ollama API"]:::external
    end

    %% CI/CD Workflows
    subgraph "CI/CD & Workflows"
        direction TB
        CI1["basic-tests-*.yml"]:::ci
        CI2["pep8-linter.yml"]:::ci
        CI3["check-links.yml"]:::ci
        CI4["check-spelling-errors.yml"]:::ci
    end

    %% Environment Setup
    subgraph "Environment & Setup"
        direction TB
        Env1["setup/"]:::env
        Req["requirements.txt"]:::env
        Pyproj["pyproject.toml"]:::env
        Pixi["pixi.toml"]:::env
    end

    %% Connections
    Data1 -->|"reads"| CH02
    Data1 -->|"reads"| CH03
    Data1 -->|"reads"| CH04
    Data1 -->|"reads"| CH05
    Data1 -->|"reads"| CH06
    Data1 -->|"reads"| CH07

    CH01 -->|"imports"| Utils
    CH02 -->|"imports"| MD02
    CH03 -->|"imports"| MD03
    CH04 -->|"imports"| MD04
    CH05 -->|"imports"| MD05
    CH06 -->|"imports"| MD06
    CH07 -->|"imports"| MD07
    CH05 -->|"runs"| Generate
    CH07 -->|"runs"| Generate

    Utils -->|"uses"| PT
    MD02 -->|"uses"| PT
    MD03 -->|"uses"| PT
    MD04 -->|"uses"| PT
    MD05 -->|"uses"| PT
    MD06 -->|"uses"| PT
    MD07 -->|"uses"| PT

    Generate -->|"loads weights"| Data2
    MD07 -->|"calls"| OA
    MD07 -->|"calls"| OL
    MD05 -->|"loads HF models"| HF
    MD06 -->|"loads HF models"| HF

    CI1 -->|"tests"| Utils
    CI1 -->|"tests"| MD02
    CI1 -->|"tests"| MD03
    CI1 -->|"tests"| MD04
    CI1 -->|"tests"| MD05
    CI1 -->|"tests"| MD06
    CI1 -->|"tests"| MD07
    CI1 -->|"tests"| CH01
    CI1 -->|"tests"| CH02
    CI1 -->|"tests"| CH03
    CI1 -->|"tests"| CH04
    CI1 -->|"tests"| CH05
    CI1 -->|"tests"| CH06
    CI1 -->|"tests"| CH07

    CI2 -->|"lints"| Utils
    CI2 -->|"lints"| CH01

    Env1 -->|"provides"| CH01
    Env1 -->|"provides"| Utils

    %% Click Events
    click Utils "https://github.com/jj-devhub/llms-from-scratch/blob/main/pkg/llms_from_scratch/utils.py"
    click MD02 "https://github.com/jj-devhub/llms-from-scratch/blob/main/pkg/llms_from_scratch/ch02.py"
    click MD03 "https://github.com/jj-devhub/llms-from-scratch/blob/main/pkg/llms_from_scratch/ch03.py"
    click MD04 "https://github.com/jj-devhub/llms-from-scratch/blob/main/pkg/llms_from_scratch/ch04.py"
    click MD05 "https://github.com/jj-devhub/llms-from-scratch/blob/main/pkg/llms_from_scratch/ch05.py"
    click MD06 "https://github.com/jj-devhub/llms-from-scratch/blob/main/pkg/llms_from_scratch/ch06.py"
    click MD07 "https://github.com/jj-devhub/llms-from-scratch/blob/main/pkg/llms_from_scratch/ch07.py"
    click Generate "https://github.com/jj-devhub/llms-from-scratch/blob/main/pkg/llms_from_scratch/generate.py"
    click KV1 "https://github.com/jj-devhub/llms-from-scratch/tree/main/pkg/llms_from_scratch/kv_cache/"
    click KV2 "https://github.com/jj-devhub/llms-from-scratch/tree/main/pkg/llms_from_scratch/kv_cache_batched/"
    click CH01 "https://github.com/jj-devhub/llms-from-scratch/tree/main/ch01/"
    click CH02 "https://github.com/jj-devhub/llms-from-scratch/tree/main/ch02/01_main-chapter-code/"
    click CH03 "https://github.com/jj-devhub/llms-from-scratch/tree/main/ch03/01_main-chapter-code/"
    click CH04 "https://github.com/jj-devhub/llms-from-scratch/tree/main/ch04/01_main-chapter-code/"
    click CH05 "https://github.com/jj-devhub/llms-from-scratch/tree/main/ch05/01_main-chapter-code/"
    click CH06 "https://github.com/jj-devhub/llms-from-scratch/tree/main/ch06/01_main-chapter-code/"
    click CH07 "https://github.com/jj-devhub/llms-from-scratch/tree/main/ch07/01_main-chapter-code/"
    click APPA "https://github.com/jj-devhub/llms-from-scratch/tree/main/appendix-A/01_main-chapter-code/"
    click APPD "https://github.com/jj-devhub/llms-from-scratch/tree/main/appendix-D/01_main-chapter-code/"
    click APPE "https://github.com/jj-devhub/llms-from-scratch/tree/main/appendix-E/01_main-chapter-code/"
    click Env1 "https://github.com/jj-devhub/llms-from-scratch/tree/main/setup/"
    click Req "https://github.com/jj-devhub/llms-from-scratch/blob/main/requirements.txt"
    click Pyproj "https://github.com/jj-devhub/llms-from-scratch/blob/main/pyproject.toml"
    click Pixi "https://github.com/jj-devhub/llms-from-scratch/blob/main/pixi.toml"
    click CI1 "https://github.com/jj-devhub/llms-from-scratch/blob/main/.github/workflows/basic-tests-*.yml"
    click CI2 "https://github.com/jj-devhub/llms-from-scratch/blob/main/.github/workflows/pep8-linter.yml"
    click CI3 "https://github.com/jj-devhub/llms-from-scratch/blob/main/.github/workflows/check-links.yml"
    click CI4 "https://github.com/jj-devhub/llms-from-scratch/blob/main/.github/workflows/check-spelling-errors.yml"

    %% Styles
    classDef data fill:#ffffb3,stroke:#333,stroke-width:1px;
    classDef library fill:#ADD8E6,stroke:#333,stroke-width:1px;
    classDef notebooks fill:#90EE90,stroke:#333,stroke-width:1px;
    classDef external fill:#d3d3d3,stroke:#333,stroke-width:1px;
    classDef ci fill:#ffcccb,stroke:#333,stroke-width:1px;
    classDef env fill:#ffe4b5,stroke:#333,stroke-width:1px;
```
