# hmdrez47-star/minigpt architecture

> Architecture diagram of the GitHub repository hmdrez47-star/minigpt, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-11-21). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/hmdrez47-star/minigpt
- Repository: https://github.com/hmdrez47-star/minigpt
- Video tour (about a minute): https://gitdiagram.com/hmdrez47-star/minigpt/video

## Overview

MiniGPT is a byte-level text-generation product: users provide UTF-8 prompt bytes, the decoder-only Transformer predicts more bytes, and the web/API layer returns generated text. Separate training reads text or medical-comment CSV data, trains the model, and writes a checkpoint consumed by serving or the CLI generator. The README documents additional corpus-cleaning and advanced tokenizer capabilities; excerpts do not show their callers or full wiring, so those modules are represented without speculative links. Operations infrastructure is omitted from this runtime-focused view.

Main entry points: [`src/serving/app.py`](https://github.com/hmdrez47-star/minigpt/blob/main/src/serving/app.py) (FastAPI service), [`train_mini.py`](https://github.com/hmdrez47-star/minigpt/blob/main/train_mini.py) (Training runner), [`data_pipeline.py`](https://github.com/hmdrez47-star/minigpt/blob/main/data_pipeline.py) (Clean and pack), [`generate_text.py`](https://github.com/hmdrez47-star/minigpt/blob/main/generate_text.py) (CLI generator)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_data["Corpus preparation"]
  node_corpus_prep["Clean and pack<br/>[data_pipeline.py]"]
  node_corpus_stats["Corpus statistics<br/>[data_pipeline.py]"]
end

subgraph group_tokens["Tokenization"]
  node_char_bpe["Character and BPE<br/>[tokenizer1.py]"]
  node_production_tok["Production tokenizer<br/>[tokenizer2.py]"]
  node_byte_input["UTF-8 bytes<br/>[train_mini.py]"]
end

subgraph group_model["Model and training"]
  node_training_entry["Training runner<br/>[train_mini.py]"]
  node_transformer["Decoder Transformer<br/>[mini_gpt.py]"]
  node_checkpoint[("Model checkpoint<br/>[train_mini.py]")]
end

subgraph group_access["Generation access"]
  node_api["FastAPI service<br/>[app.py]"]
  node_web_page["Generation page<br/>[app.py]"]
  node_cli["CLI generator<br/>[generate_text.py]"]
  node_metrics["Prometheus metrics<br/>[app.py]"]
end

node_user(("Prompt user"))
node_corpus["Text corpus"]
node_doctor_data["Medical comments"]

node_corpus -.->|"processes"| node_corpus_prep
node_doctor_data -->|"loads comments"| node_training_entry
node_corpus -->|"loads text"| node_training_entry
node_corpus_prep -->|"computes"| node_corpus_stats
node_training_entry -->|"encodes bytes"| node_byte_input
node_training_entry -->|"trains"| node_transformer
node_training_entry -->|"saves"| node_checkpoint
node_user -->|"submits prompt"| node_api
node_api -->|"loads"| node_checkpoint
node_api -->|"generates with"| node_transformer
node_api -->|"renders"| node_web_page
node_api -->|"exposes"| node_metrics
node_user -->|"requests generation"| node_cli
node_cli -->|"loads"| node_checkpoint
node_cli -->|"generates with"| node_transformer
node_production_tok -.->|"tokenizes"| node_corpus_prep
node_char_bpe -.->|"tokenizes"| node_corpus_prep

click node_doctor_data "https://github.com/hmdrez47-star/minigpt/blob/main/data/raw/doctor_comments.csv"
click node_corpus_prep "https://github.com/hmdrez47-star/minigpt/blob/main/data_pipeline.py"
click node_corpus_stats "https://github.com/hmdrez47-star/minigpt/blob/main/data_pipeline.py"
click node_char_bpe "https://github.com/hmdrez47-star/minigpt/blob/main/tokenizer1.py"
click node_production_tok "https://github.com/hmdrez47-star/minigpt/blob/main/tokenizer2.py"
click node_byte_input "https://github.com/hmdrez47-star/minigpt/blob/main/train_mini.py"
click node_training_entry "https://github.com/hmdrez47-star/minigpt/blob/main/train_mini.py"
click node_transformer "https://github.com/hmdrez47-star/minigpt/blob/main/mini_gpt.py"
click node_checkpoint "https://github.com/hmdrez47-star/minigpt/blob/main/train_mini.py"
click node_api "https://github.com/hmdrez47-star/minigpt/blob/main/src/serving/app.py"
click node_web_page "https://github.com/hmdrez47-star/minigpt/blob/main/src/serving/app.py"
click node_cli "https://github.com/hmdrez47-star/minigpt/blob/main/generate_text.py"
click node_metrics "https://github.com/hmdrez47-star/minigpt/blob/main/src/serving/app.py"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_corpus_prep,node_corpus_stats,node_user toneBlue
class node_char_bpe,node_production_tok,node_byte_input toneAmber
class node_training_entry,node_transformer,node_checkpoint toneMint
class node_api,node_web_page,node_cli,node_metrics toneRose
class node_corpus toneIndigo
class node_doctor_data toneTeal
```

## Components

### Corpus preparation

- [Clean and pack](https://github.com/hmdrez47-star/minigpt/blob/main/data_pipeline.py): `data_pipeline.py`
- [Corpus statistics](https://github.com/hmdrez47-star/minigpt/blob/main/data_pipeline.py): `data_pipeline.py`

### Tokenization

- [Character and BPE](https://github.com/hmdrez47-star/minigpt/blob/main/tokenizer1.py): `tokenizer1.py`
- [Production tokenizer](https://github.com/hmdrez47-star/minigpt/blob/main/tokenizer2.py): `tokenizer2.py`
- [UTF-8 bytes](https://github.com/hmdrez47-star/minigpt/blob/main/train_mini.py): `train_mini.py`

### Model and training

- [Training runner](https://github.com/hmdrez47-star/minigpt/blob/main/train_mini.py): `train_mini.py`
- [Decoder Transformer](https://github.com/hmdrez47-star/minigpt/blob/main/mini_gpt.py): `mini_gpt.py`
- [Model checkpoint](https://github.com/hmdrez47-star/minigpt/blob/main/train_mini.py): `train_mini.py`

### Generation access

- [FastAPI service](https://github.com/hmdrez47-star/minigpt/blob/main/src/serving/app.py): `src/serving/app.py`
- [Generation page](https://github.com/hmdrez47-star/minigpt/blob/main/src/serving/app.py): `src/serving/app.py`
- [CLI generator](https://github.com/hmdrez47-star/minigpt/blob/main/generate_text.py): `generate_text.py`
- [Prometheus metrics](https://github.com/hmdrez47-star/minigpt/blob/main/src/serving/app.py): `src/serving/app.py`

### Other components

- **Prompt user**
- **Text corpus**
- [Medical comments](https://github.com/hmdrez47-star/minigpt/blob/main/data/raw/doctor_comments.csv): `data/raw/doctor_comments.csv`

## Connections

- Text corpus → Clean and pack: processes (evidence: `README.md`)
- Medical comments → Training runner: loads comments (evidence: `train_mini.py`)
- Text corpus → Training runner: loads text (evidence: `train_mini.py`)
- Clean and pack → Corpus statistics: computes (evidence: `data_pipeline.py`)
- Training runner → UTF-8 bytes: encodes bytes (evidence: `train_mini.py`)
- Training runner → Decoder Transformer: trains (evidence: `train_mini.py`)
- Training runner → Model checkpoint: saves (evidence: `train_mini.py`)
- Prompt user → FastAPI service: submits prompt (evidence: `README.md`)
- FastAPI service → Model checkpoint: loads (evidence: `src/serving/app.py`)
- FastAPI service → Decoder Transformer: generates with (evidence: `src/serving/app.py`)
- FastAPI service → Generation page: renders (evidence: `src/serving/app.py`)
- FastAPI service → Prometheus metrics: exposes (evidence: `src/serving/app.py`)
- Prompt user → CLI generator: requests generation (evidence: `generate_text.py`)
- CLI generator → Model checkpoint: loads (evidence: `generate_text.py`)
- CLI generator → Decoder Transformer: generates with (evidence: `generate_text.py`)
- Production tokenizer → Clean and pack: tokenizes
- Character and BPE → Clean and pack: tokenizes
