# shreyasgaikwad-04/minigpt architecture

> Architecture diagram of the GitHub repository shreyasgaikwad-04/minigpt, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-09-26). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/shreyasgaikwad-04/minigpt
- Repository: https://github.com/shreyasgaikwad-04/minigpt

## Overview

MiniGPT is an educational decoder-only language model, trained on a small TinyStories subset and used to generate text from saved checkpoints. The training workflow fetches examples, fits a tokenizer, builds batches, optimizes the model with next-token loss, and saves checkpoints; generation reloads a checkpoint and tokenizer, then autoregressively samples tokens for a prompt. The model’s documented internal stages are represented within its implementation module. Training and generation source were sampled; the README documents the broader workflow, while details beyond those excerpts are not inferred.

Main entry points: [`src/train.py`](https://github.com/shreyasgaikwad-04/minigpt/blob/main/src/train.py) (Training loop), [`src/generate.py`](https://github.com/shreyasgaikwad-04/minigpt/blob/main/src/generate.py) (Generation loop), [`src/model.py`](https://github.com/shreyasgaikwad-04/minigpt/blob/main/src/model.py) (MiniGPT), [`src/data.py`](https://github.com/shreyasgaikwad-04/minigpt/blob/main/src/data.py) (Fetch and batch data)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_data["Data preparation"]
  node_dataset["Fetch and batch data<br/>[data.py]"]
  node_tokenizer["Tokenizer<br/>[tokenizer.py]"]
  node_vocabulary["Saved vocabulary<br/>[train.py]"]
end

subgraph group_model["Transformer model"]
  node_model_core["MiniGPT<br/>[model.py]"]
  node_embeddings["Token and position embeddings<br/>[model.py]"]
  node_block["Transformer blocks<br/>[model.py]"]
  node_attention["Causal attention<br/>[model.py]"]
  node_feedforward["Feed-forward layers<br/>[model.py]"]
  node_logits["Token logits<br/>[model.py]"]
end

subgraph group_train["Training"]
  node_config["Training config<br/>[tiny.yaml]"]
  node_trainer["Training loop<br/>[train.py]"]
  node_loss["Next-token loss<br/>[train.py]"]
  node_checkpoint["Model checkpoint<br/>[train.py]"]
  node_metrics["Training metrics<br/>[utils.py]"]
end

subgraph group_generate["Generation"]
  node_generator["Generation loop<br/>[generate.py]"]
  node_text["Generated text<br/>[generate.py]"]
end

node_user(("User"))
node_tinystories[("TinyStories")]

node_user -->|"starts training"| node_trainer
node_user -->|"provides prompt"| node_generator
node_trainer -->|"reads config"| node_config
node_trainer -->|"requests examples"| node_dataset
node_dataset -->|"loads subset"| node_tinystories
node_trainer -->|"fits vocabulary"| node_tokenizer
node_trainer -->|"saves vocabulary"| node_vocabulary
node_trainer -->|"builds batches"| node_dataset
node_trainer -->|"trains model"| node_model_core
node_model_core -->|"embeds IDs"| node_embeddings
node_model_core -->|"applies blocks"| node_block
node_block -->|"computes attention"| node_attention
node_block -->|"applies feed-forward"| node_feedforward
node_model_core -->|"produces logits"| node_logits
node_trainer -->|"computes loss"| node_loss
node_loss -->|"scores predictions"| node_logits
node_trainer -->|"saves checkpoint"| node_checkpoint
node_trainer -->|"saves metrics"| node_metrics
node_generator -->|"loads weights"| node_checkpoint
node_generator -->|"loads tokenizer"| node_vocabulary
node_generator -->|"encodes prompt"| node_tokenizer
node_generator -->|"queries next token"| node_model_core
node_generator -->|"decodes tokens"| node_tokenizer
node_generator -->|"returns text"| node_text
node_text -->|"displays result"| node_user

click node_dataset "https://github.com/shreyasgaikwad-04/minigpt/blob/main/src/data.py"
click node_tokenizer "https://github.com/shreyasgaikwad-04/minigpt/blob/main/src/tokenizer.py"
click node_vocabulary "https://github.com/shreyasgaikwad-04/minigpt/blob/main/src/train.py"
click node_config "https://github.com/shreyasgaikwad-04/minigpt/blob/main/configs/tiny.yaml"
click node_trainer "https://github.com/shreyasgaikwad-04/minigpt/blob/main/src/train.py"
click node_loss "https://github.com/shreyasgaikwad-04/minigpt/blob/main/src/train.py"
click node_model_core "https://github.com/shreyasgaikwad-04/minigpt/blob/main/src/model.py"
click node_embeddings "https://github.com/shreyasgaikwad-04/minigpt/blob/main/src/model.py"
click node_block "https://github.com/shreyasgaikwad-04/minigpt/blob/main/src/model.py"
click node_attention "https://github.com/shreyasgaikwad-04/minigpt/blob/main/src/model.py"
click node_feedforward "https://github.com/shreyasgaikwad-04/minigpt/blob/main/src/model.py"
click node_logits "https://github.com/shreyasgaikwad-04/minigpt/blob/main/src/model.py"
click node_checkpoint "https://github.com/shreyasgaikwad-04/minigpt/blob/main/src/train.py"
click node_metrics "https://github.com/shreyasgaikwad-04/minigpt/blob/main/src/utils.py"
click node_generator "https://github.com/shreyasgaikwad-04/minigpt/blob/main/src/generate.py"
click node_text "https://github.com/shreyasgaikwad-04/minigpt/blob/main/src/generate.py"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_dataset,node_tokenizer,node_vocabulary,node_user toneBlue
class node_model_core,node_embeddings,node_block,node_attention,node_feedforward,node_logits,node_tinystories toneAmber
class node_config,node_trainer,node_loss,node_checkpoint,node_metrics toneMint
class node_generator,node_text toneRose
```

## Components

### Data preparation

- [Fetch and batch data](https://github.com/shreyasgaikwad-04/minigpt/blob/main/src/data.py): `src/data.py`
- [Tokenizer](https://github.com/shreyasgaikwad-04/minigpt/blob/main/src/tokenizer.py): `src/tokenizer.py`
- [Saved vocabulary](https://github.com/shreyasgaikwad-04/minigpt/blob/main/src/train.py): `src/train.py`

### Transformer model

- [MiniGPT](https://github.com/shreyasgaikwad-04/minigpt/blob/main/src/model.py): `src/model.py`
- [Token and position embeddings](https://github.com/shreyasgaikwad-04/minigpt/blob/main/src/model.py): `src/model.py`
- [Transformer blocks](https://github.com/shreyasgaikwad-04/minigpt/blob/main/src/model.py): `src/model.py`
- [Causal attention](https://github.com/shreyasgaikwad-04/minigpt/blob/main/src/model.py): `src/model.py`
- [Feed-forward layers](https://github.com/shreyasgaikwad-04/minigpt/blob/main/src/model.py): `src/model.py`
- [Token logits](https://github.com/shreyasgaikwad-04/minigpt/blob/main/src/model.py): `src/model.py`

### Training

- [Training config](https://github.com/shreyasgaikwad-04/minigpt/blob/main/configs/tiny.yaml): `configs/tiny.yaml`
- [Training loop](https://github.com/shreyasgaikwad-04/minigpt/blob/main/src/train.py): `src/train.py`
- [Next-token loss](https://github.com/shreyasgaikwad-04/minigpt/blob/main/src/train.py): `src/train.py`
- [Model checkpoint](https://github.com/shreyasgaikwad-04/minigpt/blob/main/src/train.py): `src/train.py`
- [Training metrics](https://github.com/shreyasgaikwad-04/minigpt/blob/main/src/utils.py): `src/utils.py`

### Generation

- [Generation loop](https://github.com/shreyasgaikwad-04/minigpt/blob/main/src/generate.py): `src/generate.py`
- [Generated text](https://github.com/shreyasgaikwad-04/minigpt/blob/main/src/generate.py): `src/generate.py`

### Other components

- **User**
- **TinyStories**

## Connections

- User → Training loop: starts training
- User → Generation loop: provides prompt
- Training loop → Training config: reads config
- Training loop → Fetch and batch data: requests examples
- Fetch and batch data → TinyStories: loads subset
- Training loop → Tokenizer: fits vocabulary
- Training loop → Saved vocabulary: saves vocabulary
- Training loop → Fetch and batch data: builds batches
- Training loop → MiniGPT: trains model
- MiniGPT → Token and position embeddings: embeds IDs
- MiniGPT → Transformer blocks: applies blocks
- Transformer blocks → Causal attention: computes attention
- Transformer blocks → Feed-forward layers: applies feed-forward
- MiniGPT → Token logits: produces logits
- Training loop → Next-token loss: computes loss
- Next-token loss → Token logits: scores predictions
- Training loop → Model checkpoint: saves checkpoint
- Training loop → Training metrics: saves metrics
- Generation loop → Model checkpoint: loads weights
- Generation loop → Saved vocabulary: loads tokenizer
- Generation loop → Tokenizer: encodes prompt
- Generation loop → MiniGPT: queries next token
- Generation loop → Tokenizer: decodes tokens
- Generation loop → Generated text: returns text
- Generated text → User: displays result