How jpvercosa/machado-transformer works
This repository trains and runs a character-level Transformer that generates text in the style of Machado de Assis. The main workflow is to read a selected corpus, encode and split it, train the model, save a checkpoint, then load that checkpoint to generate and decode text. The model includes attention heads, multi-head attention, feed-forward layers, and Transformer blocks. The graph reflects the sampled scripts and README; it does not assume details of unsampled notebooks or checkpoint contents.
Main entry points: train.py Training command · inference.py Inference command · nanoGPT_model.py GPT language model · utils.py Text and training utilities
Components 13
Command-line workflows
- Training command
train.py - Inference command
inference.py
Corpus and utilities
- Text and training utilities
utils.py
Transformer model
- Model configuration
config.py - GPT language model
nanoGPT_model.py - Transformer block
nanoGPT_model.py - Multi-head attention
nanoGPT_model.py - Attention head
nanoGPT_model.py - Feed-forward layer
nanoGPT_model.py
Text and checkpoints
- Text corpora
data - Model checkpoints
models
Other components
- CLI user
- Generated text
Connections 20
- CLI user → Training command: starts training
README.md - CLI user → Inference command: requests generation
README.md - Training command → Text corpora: reads corpus
train.py - Training command → Text and training utilities: prepares data
train.py - Training command → Model configuration: uses settings
train.py - Training command → GPT language model: trains model
train.py - Training command → Text and training utilities: evaluates batches
train.py - Training command → Model checkpoints: saves checkpoint
train.py - Training command → Generated text: prints sample
train.py - Inference command → Text corpora: reads corpus
inference.py - Inference command → Text and training utilities: decodes text
inference.py - Inference command → Model checkpoints: loads checkpoint
inference.py - Inference command → Model configuration: uses device
inference.py - Inference command → GPT language model: generates tokens
inference.py - Inference command → Generated text: prints text
inference.py - GPT language model → Model configuration: reads parameters
nanoGPT_model.py - GPT language model → Transformer block: applies blocks
nanoGPT_model.py - Transformer block → Multi-head attention: applies attention
nanoGPT_model.py - Transformer block → Feed-forward layer: applies feed-forward
nanoGPT_model.py - Multi-head attention → Attention head: runs heads
nanoGPT_model.py