How jruttan1/mini-gpt works
Project type and purpose
• This is a small-scale, research-oriented machine-learning project: a character-level GPT (transformer) language model implemented in Python.
• Its goal is to enable quick experimentation with architecture and hyperparameters on the Tiny Shakespeare dataset, producing sample text outputs.
File structure
• Top level:
• .idea/ – IDE configs (irrelevant to runtime)
• gpt_model.py – primary training & generation script
• gpt_model2.py – variant script for architectural/hyperparameter comparisons
• mini_gpt.ipynb – educational Jupyter notebook version (bigram model, in-notebook training & visualization)
• input.txt – raw Tiny Shakespeare text dataset
• output*.txt – sample text outputs
• No separate “data/” or “src/” subdirectories—everything lives in root, emphasizing simplicity.
README for additional insights
• Confirms this is purely a script-driven experiment: data loading, model definition (embedding, multi-head self-attention, feed-forward blocks), training loop, evaluation and generation.
• Hyperparameters are hard-coded near the top of each script or notebook cell for easy tweaking.