How vizuaraailabs/nano-gpt-oss works
Nano-GPT-OSS is a language-model project with GPT-OSS and GPT-2 architectures, TinyStories training data preparation, and prompt-based text generation. The documented training workflow is not fully wired in the sampled entry point: train.py builds and saves a model and generates text, while training/trainer.py contains a separate optimization and evaluation loop. Edges reflect sampled calls; other README capabilities are represented where the tree supports them.
Main entry points: gptoss.py GPT-OSS transformer · inference.py Text generation · trainer.py Trainer and evaluator · data_loader.py Dataset loader
Components 12
Data preparation
- Dataset loader
training/data_loader.py - Tokenizer
architecture/tokenizer.py
Model architectures
- GPT-OSS transformer
architecture/gptoss.py - Attention and RoPE
architecture/gptoss.py - MoE SwiGLU block
architecture/gptoss.py - GPT-2 transformer
architecture/gpt2.py
Training and evaluation
- Train entry point
train.py - Trainer and evaluator
training/trainer.py
Text generation
- Text generation
inference.py
Other components
- User
- Hugging Face datasets
- Weights & Biases
Connections 14
- Dataset loader → Hugging Face datasets: loads stories
- Dataset loader → Tokenizer: encodes text
- Train entry point → Dataset loader: imports loaders
- Train entry point → GPT-OSS transformer: constructs model
- Train entry point → Text generation: generates text
- GPT-OSS transformer → Attention and RoPE: uses attention
- GPT-OSS transformer → MoE SwiGLU block: uses MoE
- User → Text generation: provides prompt
- Text generation → Tokenizer: encodes and decodes
- Text generation → GPT-OSS transformer: requests logits
- Trainer and evaluator → Weights & Biases: logs metrics
- Trainer and evaluator → Text generation: generates sample
- Trainer and evaluator → GPT-OSS transformer: trains and evaluates
- Trainer and evaluator → Dataset loader: consumes loaders