How dberardi99/machine-learning-roadmap-zero-to-hero works
Project type and purpose
• This is an educational “roadmap” repository for learning core ML concepts by building small frameworks from scratch.
• It is organized into two primary modules:
a. build-LLMs-from-scratch – guided exercises for text‐data processing, tokenization, and eventually LLM components.
b. micrograd – a tiny automatic‐differentiation engine illustrating backpropagation.
File structure
• Top‐level items:
• LICENSE, .gitignore, README.md
• build-LLMs-from-scratch/
• micrograd/
• Within build-LLMs-from-scratch/1-working-with-text-data/:
• Core code: tokenizer.py, 1-working-with-text-data-notes-and-tests.py
• Utilities: utils/gpt2-vocab.json, utils/gpt2-merges.txt, utils/the-verdict.txt
• Tests: tests/test.py
• Supporting images
• Within micrograd/:
• Core engine: micrograd.py
• Demo & tests: micrograd-notes-and-tests.py
• Supporting images
README
• Confirms the repo is a step-by-step machine‐learning roadmap.
a. Learning Modules (two major subsystems)
1. Text‐Data Module (build-LLMs-from-scratch/1-working-with-text-data)
2. Automatic‐Diff Module (micrograd/)
b. Data Artifacts
– Raw text files (the-verdict.txt)
– Vocabulary and merge tables (gpt2-vocab.json, gpt2-merges.txt)
c. Processing Components
– Tokenizer (tokenizer.py)
– Core micrograd engine (micrograd.py)
d. Test/Validation Harness
– tests/test.py in each module
– Interactive/demo notebooks (*-notes-and-tests.py)
e. Assets
– Images illustrating concepts
Relationships and interactions
• Raw text → Tokenizer → token sequences
• Tokens + merges/vocab → demonstration scripts → tests
• Micrograd core → computational graph → gradient computations → demo scripts/tests
• Each module stands alone but follows the same pattern: Data/Assets → Core Logic → Demos/Tests
Architectural patterns and principles
• Modular separation: each concept in its own directory
• Single responsibility: tokenizer only handles text; micrograd only handles autodiff
• Test-driven learning: each step accompanied by tests and notes
• Data‐flow orientation: clear pipeline from raw data → processing → validation
Technologies/frameworks
• Pure Python 3 scripts (no external ML frameworks)
• JSON/text assets for vocabulary
• Basic file‐based testing
Detail without overcomplicating
• every folder as a component
• How utilities feed into core scripts
• Link core scripts to their corresponding tests/demos
• Do not merge modules—keep the two learning tracks side by side