# rajcoder123-noob/transformer architecture

> Architecture diagram of the GitHub repository rajcoder123-noob/transformer, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-11-15). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/rajcoder123-noob/transformer
- Repository: https://github.com/rajcoder123-noob/transformer

## Overview

MiniGPT is an educational language-modeling project with two runnable paths: a character-level Bigram baseline and a GPT-2-BPE decoder-only Transformer. The GPT path prepares a corpus, tokenizes and batches it, trains and evaluates the model, saves checkpoints, then generates text from a prompt. The README also documents OpenWebText ingestion and checkpointing. Source excerpts show the implementations and their calls; the documented corpus artifact is not present in the supplied tree, so it has no repository path.

Main entry points: [`gpt.py`](https://github.com/rajcoder123-noob/transformer/blob/main/gpt.py) (Training Loop), [`bigram.py`](https://github.com/rajcoder123-noob/transformer/blob/main/bigram.py) (Character Bigram), [`download_data.py`](https://github.com/rajcoder123-noob/transformer/blob/main/download_data.py) (Corpus Downloader), [`generated.txt`](https://github.com/rajcoder123-noob/transformer/blob/main/generated.txt) (Generated Text)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_data["Corpus Preparation"]
  node_downloader["Corpus Downloader<br/>[download_data.py]"]
  node_corpus["Training Corpus"]
end

subgraph group_baseline["Bigram Baseline"]
  node_char_model["Character Bigram<br/>[bigram.py]"]
  node_char_codec["Character Codec<br/>[bigram.py]"]
end

subgraph group_gpt["GPT Transformer"]
  node_bpe["GPT-2 BPE<br/>[gpt.py]"]
  node_splitter["Train Validation Split<br/>[gpt.py]"]
  node_batcher["Batch Loader<br/>[gpt.py]"]
  node_attention["Masked Attention<br/>[gpt.py]"]
  node_feedforward["Feed Forward<br/>[gpt.py]"]
  node_transformer["GPT Language Model<br/>[gpt.py]"]
  node_evaluation["Loss Evaluation<br/>[gpt.py]"]
  node_trainer["Training Loop<br/>[gpt.py]"]
end

subgraph group_outputs["Model Outputs"]
  node_prompt["Generation Prompt"]
  node_checkpoint[("Model Checkpoints<br/>[gpt.py]")]
  node_generation["Text Generation<br/>[gpt.py]"]
  node_generated_text["Generated Text<br/>[generated.txt]"]
end

node_user(("User"))
node_openwebtext[("OpenWebText")]

node_openwebtext -->|"streams documents"| node_downloader
node_downloader -->|"writes corpus"| node_corpus
node_corpus -->|"loads text"| node_char_model
node_corpus -->|"loads text"| node_bpe
node_char_model -->|"encodes characters"| node_char_codec
node_char_model -->|"trains and samples"| node_char_model
node_user -->|"runs preparation"| node_downloader
node_user -->|"runs baseline"| node_char_model
node_user -->|"starts training"| node_trainer
node_bpe -->|"encodes tokens"| node_splitter
node_splitter -->|"provides splits"| node_batcher
node_batcher -->|"provides batches"| node_trainer
node_trainer -->|"evaluates loss"| node_evaluation
node_trainer -->|"trains model"| node_transformer
node_transformer -->|"uses attention"| node_attention
node_transformer -->|"uses feed forward"| node_feedforward
node_trainer -->|"saves weights"| node_checkpoint
node_user -->|"supplies prompt"| node_prompt
node_prompt -->|"starts generation"| node_generation
node_generation -->|"samples next tokens"| node_transformer
node_generation -->|"decodes tokens"| node_bpe
node_generation -->|"writes output"| node_generated_text

click node_downloader "https://github.com/rajcoder123-noob/transformer/blob/main/download_data.py"
click node_char_model "https://github.com/rajcoder123-noob/transformer/blob/main/bigram.py"
click node_char_codec "https://github.com/rajcoder123-noob/transformer/blob/main/bigram.py"
click node_bpe "https://github.com/rajcoder123-noob/transformer/blob/main/gpt.py"
click node_splitter "https://github.com/rajcoder123-noob/transformer/blob/main/gpt.py"
click node_batcher "https://github.com/rajcoder123-noob/transformer/blob/main/gpt.py"
click node_attention "https://github.com/rajcoder123-noob/transformer/blob/main/gpt.py"
click node_feedforward "https://github.com/rajcoder123-noob/transformer/blob/main/gpt.py"
click node_transformer "https://github.com/rajcoder123-noob/transformer/blob/main/gpt.py"
click node_evaluation "https://github.com/rajcoder123-noob/transformer/blob/main/gpt.py"
click node_trainer "https://github.com/rajcoder123-noob/transformer/blob/main/gpt.py"
click node_checkpoint "https://github.com/rajcoder123-noob/transformer/blob/main/gpt.py"
click node_generation "https://github.com/rajcoder123-noob/transformer/blob/main/gpt.py"
click node_generated_text "https://github.com/rajcoder123-noob/transformer/blob/main/generated.txt"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_downloader,node_corpus,node_user toneBlue
class node_char_model,node_char_codec,node_openwebtext toneAmber
class node_bpe,node_splitter,node_batcher,node_attention,node_feedforward,node_transformer,node_evaluation,node_trainer toneMint
class node_prompt,node_checkpoint,node_generation,node_generated_text toneRose
```

## Components

### Corpus Preparation

- [Corpus Downloader](https://github.com/rajcoder123-noob/transformer/blob/main/download_data.py): `download_data.py`
- **Training Corpus**

### Bigram Baseline

- [Character Bigram](https://github.com/rajcoder123-noob/transformer/blob/main/bigram.py): `bigram.py`
- [Character Codec](https://github.com/rajcoder123-noob/transformer/blob/main/bigram.py): `bigram.py`

### GPT Transformer

- [GPT-2 BPE](https://github.com/rajcoder123-noob/transformer/blob/main/gpt.py): `gpt.py`
- [Train Validation Split](https://github.com/rajcoder123-noob/transformer/blob/main/gpt.py): `gpt.py`
- [Batch Loader](https://github.com/rajcoder123-noob/transformer/blob/main/gpt.py): `gpt.py`
- [Masked Attention](https://github.com/rajcoder123-noob/transformer/blob/main/gpt.py): `gpt.py`
- [Feed Forward](https://github.com/rajcoder123-noob/transformer/blob/main/gpt.py): `gpt.py`
- [GPT Language Model](https://github.com/rajcoder123-noob/transformer/blob/main/gpt.py): `gpt.py`
- [Loss Evaluation](https://github.com/rajcoder123-noob/transformer/blob/main/gpt.py): `gpt.py`
- [Training Loop](https://github.com/rajcoder123-noob/transformer/blob/main/gpt.py): `gpt.py`

### Model Outputs

- **Generation Prompt**
- [Model Checkpoints](https://github.com/rajcoder123-noob/transformer/blob/main/gpt.py): `gpt.py`
- [Text Generation](https://github.com/rajcoder123-noob/transformer/blob/main/gpt.py): `gpt.py`
- [Generated Text](https://github.com/rajcoder123-noob/transformer/blob/main/generated.txt): `generated.txt`

### Other components

- **User**
- **OpenWebText**

## Connections

- OpenWebText → Corpus Downloader: streams documents (evidence: `README.md`)
- Corpus Downloader → Training Corpus: writes corpus (evidence: `README.md`)
- Training Corpus → Character Bigram: loads text (evidence: `bigram.py`)
- Training Corpus → GPT-2 BPE: loads text (evidence: `gpt.py`)
- Character Bigram → Character Codec: encodes characters (evidence: `bigram.py`)
- Character Bigram → Character Bigram: trains and samples (evidence: `bigram.py`)
- User → Corpus Downloader: runs preparation (evidence: `README.md`)
- User → Character Bigram: runs baseline (evidence: `README.md`)
- User → Training Loop: starts training (evidence: `README.md`)
- GPT-2 BPE → Train Validation Split: encodes tokens (evidence: `gpt.py`)
- Train Validation Split → Batch Loader: provides splits (evidence: `gpt.py`)
- Batch Loader → Training Loop: provides batches (evidence: `gpt.py`)
- Training Loop → Loss Evaluation: evaluates loss (evidence: `gpt.py`)
- Training Loop → GPT Language Model: trains model (evidence: `gpt.py`)
- GPT Language Model → Masked Attention: uses attention (evidence: `gpt.py`)
- GPT Language Model → Feed Forward: uses feed forward (evidence: `gpt.py`)
- Training Loop → Model Checkpoints: saves weights (evidence: `gpt.py`)
- User → Generation Prompt: supplies prompt (evidence: `README.md`)
- Generation Prompt → Text Generation: starts generation (evidence: `README.md`)
- Text Generation → GPT Language Model: samples next tokens (evidence: `gpt.py`)
- Text Generation → GPT-2 BPE: decodes tokens (evidence: `gpt.py`)
- Text Generation → Generated Text: writes output (evidence: `gpt.py`)