How infatoshi/fcc-intro-to-llms works
This is a small educational project for learning to build language models from scratch, with a data-extraction script and several Jupyter notebooks. The sampled script confirms extraction of sampled OpenWebText archives into train/validation text and a character vocabulary. Notebook internals were not available, so their precise inputs, calls, and model outputs are not inferred; the README identifies Wizard of Oz as a fallback training/validation dataset.
Main entry points: data-extract.py Corpus extractor · vocab.txt Character vocabulary
Components 9
Data preparation
- Corpus extractor
data-extract.py - Train/validation text
- Character vocabulary
vocab.txt - Wizard of Oz corpus
wizard_of_oz.txt
LLM learning
- Bigram notebook
bigram.ipynb - LLM notebook
v1.ipynb - PyTorch examples
torch-examples.ipynb
Other components
- Learner
- OpenWebText archives
Connections 3
- OpenWebText archives → Corpus extractor: reads archives
- Corpus extractor → Train/validation text: writes sampled text
- Corpus extractor → Character vocabulary: writes characters