# jruttan1/mini-gpt architecture

> Architecture diagram of the GitHub repository jruttan1/mini-gpt, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-06-22). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/jruttan1/mini-gpt
- Repository: https://github.com/jruttan1/mini-gpt

## Overview

##### Project type and purpose
- This is a small-scale, research-oriented machine-learning project: a character-level GPT (transformer) language model implemented in Python.
- Its goal is to enable quick experimentation with architecture and hyperparameters on the Tiny Shakespeare dataset, producing sample text outputs.

##### File structure
- Top level:
- `.idea/` – IDE configs (irrelevant to runtime)
- `gpt_model.py` – primary training & generation script
- `gpt_model2.py` – variant script for architectural/hyperparameter comparisons
- `mini_gpt.ipynb` – educational Jupyter notebook version (bigram model, in-notebook training & visualization)
- `input.txt` – raw Tiny Shakespeare text dataset
- `output*.txt` – sample text outputs
- No separate “data/” or “src/” subdirectories—everything lives in root, emphasizing simplicity.

##### README for additional insights
- Confirms this is purely a script-driven experiment: data loading, model definition (embedding, multi-head self-attention, feed-forward blocks), training loop, evaluation and generation.
- Hyperparameters are hard-coded near the top of each script or notebook cell for easy tweaking.

## Diagram (Mermaid)

```mermaid
flowchart TD
    subgraph "Data Layer"
        A["input.txt"]:::data
    end
    A --> B["Data Ingestion & Preprocessing"]:::runtime

    subgraph "Model Definition"
        B1["Embedding Layer"]:::runtime
        B2["Positional Encoding"]:::runtime
        B3["Transformer Blocks"]:::runtime
        B4["Output Layer"]:::runtime
    end
    B --> B1
    B1 --> B2
    B2 --> B3
    B3 --> B4

    B4 --> C["Training Loop"]:::runtime
    C --> D["Evaluation & Sampling"]:::runtime

    subgraph "Generated Outputs"
        E1["output1.txt"]:::data
        E2["output2.txt"]:::data
        E3["output3.txt"]:::data
    end
    D --> E1
    D --> E2
    D --> E3

    subgraph "Experiment Variants"
        F1["gpt_model.py"]:::code
        F2["gpt_model2.py"]:::code
        F3["mini_gpt.ipynb"]:::code
    end
    B4 --> F1
    B4 --> F2
    B4 --> F3

    G["README.md"]:::docs
    H[".idea/"]:::config

    %% Click Events
    click A "https://github.com/jruttan1/mini-gpt/blob/main/input.txt"
    click F1 "https://github.com/jruttan1/mini-gpt/blob/main/gpt_model.py"
    click F2 "https://github.com/jruttan1/mini-gpt/blob/main/gpt_model2.py"
    click F3 "https://github.com/jruttan1/mini-gpt/blob/main/mini_gpt.ipynb"
    click E1 "https://github.com/jruttan1/mini-gpt/blob/main/output1.txt"
    click E2 "https://github.com/jruttan1/mini-gpt/blob/main/output2.txt"
    click E3 "https://github.com/jruttan1/mini-gpt/blob/main/output3.txt"
    click G "https://github.com/jruttan1/mini-gpt/blob/main/README.md"
    click H "https://github.com/jruttan1/mini-gpt/tree/main/.idea/"

    %% Styles
    classDef data fill:#94c2fc,stroke:#333,stroke-width:1px
    classDef code fill:#b5e48c,stroke:#333,stroke-width:1px
    classDef runtime fill:#fca311,stroke:#333,stroke-width:1px
    classDef docs fill:#a0a0a0,stroke:#333,stroke-width:1px
    classDef config fill:#c0c0c0,stroke:#333,stroke-width:1px
```
