roiamiel1/build-deep-learning-from-scratch

How roiamiel1/build-deep-learning-from-scratch works

This repository is a project-based curriculum, not a packaged deep-learning runtime: learners implement skeletons stage by stage to build an autodiff engine and then progressively larger models. The principal learning workflow runs from scalar backpropagation through tensor operations, layers, losses and training, then branches into CNNs and attention-based models before framework and capstone projects. Major capabilities are grouped below; the sampled code confirms the tensor, MLP, convolution, framework, and capstone themes, while many later-stage details come from the README rather than inspected implementations.

Main entry points: stage_11_mlp/code.py MLP · stage_15_first_training_loop/code.py Training loop · stage_32_framework_refactor/code.py mytorch framework · stage_30_transformer/code.py Transformer models

Components 24

Autodiff foundations

Scalar autodiff stages/stage_01_scalar_values/code.py
Graph and derivatives stages/stage_02_computational_graph/code.py
Backprop engine stages/stage_05_backprop_engine/code.py
Vector and matrix ops stages/stage_06_vector_operations/code.py
Tensor engine stages/stage_08_tensor_engine/code.py

Models and training

Neuron and dense layers stages/stage_10_dense_layer/code.py
MLP stages/stage_11_mlp/code.py
Loss functions stages/stage_12_loss_functions/code.py
Batched losses stages/stage_13_loss_functions_batched/code.py
Optimizers stages/stage_14_sgd_optimizer/code.py
Training loop stages/stage_15_first_training_loop/code.py
Batch training and loader stages/stage_20_dataloader/code.py
Regularization layers stages/stage_21_regularization/code.py

Vision models

Convolution math stages/stage_24_conv2d_math/code.py
Convolution layers stages/stage_25_conv2d_implementation/code.py
CNN project stages/stage_26_cnn_project/code.py

Attention models

Attention math stages/stage_27_attention_math/code.py
Transformer models stages/stage_30_transformer/code.py
Vision Transformer stages/stage_31_vision_transformer/code.py

Framework and capstones

mytorch framework stages/stage_32_framework_refactor/code.py
MNIST capstone stages/stage_33_capstone_mnist/code.py
CIFAR-10 capstone stages/stage_34_capstone_cifar10/code.py
Transformer capstone stages/stage_35_capstone_transformer/code.py

Other components

Learner

Connections 25

Learner → Scalar autodiff: implements README.md
Scalar autodiff → Graph and derivatives: extends README.md
Graph and derivatives → Backprop engine: extends README.md
Backprop engine → Vector and matrix ops: generalizes to README.md
Vector and matrix ops → Tensor engine: generalizes to README.md
Tensor engine → Neuron and dense layers: builds on README.md
Neuron and dense layers → MLP: stacks README.md
MLP → Loss functions: trains with README.md
Loss functions → Batched losses: extends to batches README.md
MLP → Optimizers: updates with README.md
Training loop → MLP: uses README.md
Training loop → Loss functions: computes README.md
Training loop → Optimizers: steps README.md
Batch training and loader → Training loop: feeds batches README.md
Regularization layers → MLP: regularizes README.md
Convolution math → Convolution layers: informs README.md
Convolution layers → CNN project: composes README.md
Attention math → Transformer models: generalizes to README.md
Transformer models → Vision Transformer: extends to images README.md
mytorch framework → MNIST capstone: supports README.md
mytorch framework → CIFAR-10 capstone: supports README.md
mytorch framework → Transformer capstone: supports README.md
Transformer models → Transformer capstone: assembles README.md
CNN project → CIFAR-10 capstone: forms basis README.md
Vision Transformer → MNIST capstone: modeling option

Drawn by GitDiagram on November 15, 2026 from the repository's file tree, README and source files. Read as Markdown ·