# jpvercosa/machado-transformer architecture

> Architecture diagram of the GitHub repository jpvercosa/machado-transformer, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-10-05). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/jpvercosa/machado-transformer
- Repository: https://github.com/jpvercosa/machado-transformer

## Overview

This repository trains and runs a character-level Transformer that generates text in the style of Machado de Assis. The main workflow is to read a selected corpus, encode and split it, train the model, save a checkpoint, then load that checkpoint to generate and decode text. The model includes attention heads, multi-head attention, feed-forward layers, and Transformer blocks. The graph reflects the sampled scripts and README; it does not assume details of unsampled notebooks or checkpoint contents.

Main entry points: [`train.py`](https://github.com/jpvercosa/machado-transformer/blob/main/train.py) (Training command), [`inference.py`](https://github.com/jpvercosa/machado-transformer/blob/main/inference.py) (Inference command), [`nanoGPT_model.py`](https://github.com/jpvercosa/machado-transformer/blob/main/nanoGPT_model.py) (GPT language model), [`utils.py`](https://github.com/jpvercosa/machado-transformer/blob/main/utils.py) (Text and training utilities)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_entry["Command-line workflows"]
  node_train["Training command<br/>[train.py]"]
  node_infer["Inference command<br/>[inference.py]"]
end

subgraph group_prep["Corpus and utilities"]
  node_utils["Text and training utilities<br/>[utils.py]"]
end

subgraph group_model["Transformer model"]
  node_config["Model configuration<br/>[config.py]"]
  node_gpt["GPT language model<br/>[nanoGPT_model.py]"]
  node_block["Transformer block<br/>[nanoGPT_model.py]"]
  node_mha["Multi-head attention<br/>[nanoGPT_model.py]"]
  node_head["Attention head<br/>[nanoGPT_model.py]"]
  node_ffn["Feed-forward layer<br/>[nanoGPT_model.py]"]
end

subgraph group_artifacts["Text and checkpoints"]
  node_corpora["Text corpora"]
  node_checkpoints[("Model checkpoints")]
end

node_user(("CLI user"))
node_console(("Generated text"))

node_user -->|"starts training"| node_train
node_user -->|"requests generation"| node_infer
node_train -->|"reads corpus"| node_corpora
node_train -->|"prepares data"| node_utils
node_train -->|"uses settings"| node_config
node_train -->|"trains model"| node_gpt
node_train -->|"evaluates batches"| node_utils
node_train -->|"saves checkpoint"| node_checkpoints
node_train -->|"prints sample"| node_console
node_infer -->|"reads corpus"| node_corpora
node_infer -->|"decodes text"| node_utils
node_infer -->|"loads checkpoint"| node_checkpoints
node_infer -->|"uses device"| node_config
node_infer -->|"generates tokens"| node_gpt
node_infer -->|"prints text"| node_console
node_gpt -->|"reads parameters"| node_config
node_gpt -->|"applies blocks"| node_block
node_block -->|"applies attention"| node_mha
node_block -->|"applies feed-forward"| node_ffn
node_mha -->|"runs heads"| node_head

click node_train "https://github.com/jpvercosa/machado-transformer/blob/main/train.py"
click node_infer "https://github.com/jpvercosa/machado-transformer/blob/main/inference.py"
click node_utils "https://github.com/jpvercosa/machado-transformer/blob/main/utils.py"
click node_config "https://github.com/jpvercosa/machado-transformer/blob/main/config.py"
click node_gpt "https://github.com/jpvercosa/machado-transformer/blob/main/nanoGPT_model.py"
click node_block "https://github.com/jpvercosa/machado-transformer/blob/main/nanoGPT_model.py"
click node_mha "https://github.com/jpvercosa/machado-transformer/blob/main/nanoGPT_model.py"
click node_head "https://github.com/jpvercosa/machado-transformer/blob/main/nanoGPT_model.py"
click node_ffn "https://github.com/jpvercosa/machado-transformer/blob/main/nanoGPT_model.py"
click node_corpora "https://github.com/jpvercosa/machado-transformer/tree/main/data"
click node_checkpoints "https://github.com/jpvercosa/machado-transformer/tree/main/models"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_train,node_infer,node_user toneBlue
class node_utils toneAmber
class node_config,node_gpt,node_block,node_mha,node_head,node_ffn toneMint
class node_corpora,node_checkpoints toneRose
class node_console toneIndigo
```

## Components

### Command-line workflows

- [Training command](https://github.com/jpvercosa/machado-transformer/blob/main/train.py): `train.py`
- [Inference command](https://github.com/jpvercosa/machado-transformer/blob/main/inference.py): `inference.py`

### Corpus and utilities

- [Text and training utilities](https://github.com/jpvercosa/machado-transformer/blob/main/utils.py): `utils.py`

### Transformer model

- [Model configuration](https://github.com/jpvercosa/machado-transformer/blob/main/config.py): `config.py`
- [GPT language model](https://github.com/jpvercosa/machado-transformer/blob/main/nanoGPT_model.py): `nanoGPT_model.py`
- [Transformer block](https://github.com/jpvercosa/machado-transformer/blob/main/nanoGPT_model.py): `nanoGPT_model.py`
- [Multi-head attention](https://github.com/jpvercosa/machado-transformer/blob/main/nanoGPT_model.py): `nanoGPT_model.py`
- [Attention head](https://github.com/jpvercosa/machado-transformer/blob/main/nanoGPT_model.py): `nanoGPT_model.py`
- [Feed-forward layer](https://github.com/jpvercosa/machado-transformer/blob/main/nanoGPT_model.py): `nanoGPT_model.py`

### Text and checkpoints

- [Text corpora](https://github.com/jpvercosa/machado-transformer/tree/main/data): `data`
- [Model checkpoints](https://github.com/jpvercosa/machado-transformer/tree/main/models): `models`

### Other components

- **CLI user**
- **Generated text**

## Connections

- CLI user → Training command: starts training (evidence: `README.md`)
- CLI user → Inference command: requests generation (evidence: `README.md`)
- Training command → Text corpora: reads corpus (evidence: `train.py`)
- Training command → Text and training utilities: prepares data (evidence: `train.py`)
- Training command → Model configuration: uses settings (evidence: `train.py`)
- Training command → GPT language model: trains model (evidence: `train.py`)
- Training command → Text and training utilities: evaluates batches (evidence: `train.py`)
- Training command → Model checkpoints: saves checkpoint (evidence: `train.py`)
- Training command → Generated text: prints sample (evidence: `train.py`)
- Inference command → Text corpora: reads corpus (evidence: `inference.py`)
- Inference command → Text and training utilities: decodes text (evidence: `inference.py`)
- Inference command → Model checkpoints: loads checkpoint (evidence: `inference.py`)
- Inference command → Model configuration: uses device (evidence: `inference.py`)
- Inference command → GPT language model: generates tokens (evidence: `inference.py`)
- Inference command → Generated text: prints text (evidence: `inference.py`)
- GPT language model → Model configuration: reads parameters (evidence: `nanoGPT_model.py`)
- GPT language model → Transformer block: applies blocks (evidence: `nanoGPT_model.py`)
- Transformer block → Multi-head attention: applies attention (evidence: `nanoGPT_model.py`)
- Transformer block → Feed-forward layer: applies feed-forward (evidence: `nanoGPT_model.py`)
- Multi-head attention → Attention head: runs heads (evidence: `nanoGPT_model.py`)
