# roiamiel1/build-deep-learning-from-scratch architecture

> Architecture diagram of the GitHub repository roiamiel1/build-deep-learning-from-scratch, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-11-15). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/roiamiel1/build-deep-learning-from-scratch
- Repository: https://github.com/roiamiel1/build-deep-learning-from-scratch

## Overview

This repository is a project-based curriculum, not a packaged deep-learning runtime: learners implement skeletons stage by stage to build an autodiff engine and then progressively larger models. The principal learning workflow runs from scalar backpropagation through tensor operations, layers, losses and training, then branches into CNNs and attention-based models before framework and capstone projects. Major capabilities are grouped below; the sampled code confirms the tensor, MLP, convolution, framework, and capstone themes, while many later-stage details come from the README rather than inspected implementations.

Main entry points: [`stages/stage_11_mlp/code.py`](https://github.com/roiamiel1/build-deep-learning-from-scratch/blob/main/stages/stage_11_mlp/code.py) (MLP), [`stages/stage_15_first_training_loop/code.py`](https://github.com/roiamiel1/build-deep-learning-from-scratch/blob/main/stages/stage_15_first_training_loop/code.py) (Training loop), [`stages/stage_32_framework_refactor/code.py`](https://github.com/roiamiel1/build-deep-learning-from-scratch/blob/main/stages/stage_32_framework_refactor/code.py) (mytorch framework), [`stages/stage_30_transformer/code.py`](https://github.com/roiamiel1/build-deep-learning-from-scratch/blob/main/stages/stage_30_transformer/code.py) (Transformer models)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_foundations["Autodiff foundations"]
  node_scalar["Scalar autodiff<br/>[code.py]"]
  node_graph["Graph and derivatives<br/>[code.py]"]
  node_backprop["Backprop engine<br/>[code.py]"]
  node_vecmat["Vector and matrix ops<br/>[code.py]"]
  node_tensor["Tensor engine<br/>[code.py]"]
end

subgraph group_modeling["Models and training"]
  node_layers["Neuron and dense layers<br/>[code.py]"]
  node_mlp["MLP<br/>[code.py]"]
  node_losses["Loss functions<br/>[code.py]"]
  node_batched_loss["Batched losses<br/>[code.py]"]
  node_optimizers["Optimizers<br/>[code.py]"]
  node_training["Training loop<br/>[code.py]"]
  node_batchdata["Batch training and loader<br/>[code.py]"]
  node_regularization["Regularization layers<br/>[code.py]"]
end

subgraph group_vision["Vision models"]
  node_conv_math["Convolution math<br/>[code.py]"]
  node_conv_layers["Convolution layers<br/>[code.py]"]
  node_cnn["CNN project<br/>[code.py]"]
end

subgraph group_attention["Attention models"]
  node_attention_math["Attention math<br/>[code.py]"]
  node_transformer_blocks["Transformer models<br/>[code.py]"]
  node_vit["Vision Transformer<br/>[code.py]"]
end

subgraph group_integration["Framework and capstones"]
  node_framework["mytorch framework<br/>[code.py]"]
  node_mnist["MNIST capstone<br/>[code.py]"]
  node_cifar["CIFAR-10 capstone<br/>[code.py]"]
  node_language_model["Transformer capstone<br/>[code.py]"]
end

node_learner(("Learner"))

node_learner -->|"implements"| node_scalar
node_scalar -->|"extends"| node_graph
node_graph -->|"extends"| node_backprop
node_backprop -->|"generalizes to"| node_vecmat
node_vecmat -->|"generalizes to"| node_tensor
node_tensor -->|"builds on"| node_layers
node_layers -->|"stacks"| node_mlp
node_mlp -->|"trains with"| node_losses
node_losses -->|"extends to batches"| node_batched_loss
node_mlp -->|"updates with"| node_optimizers
node_training -->|"uses"| node_mlp
node_training -->|"computes"| node_losses
node_training -->|"steps"| node_optimizers
node_batchdata -->|"feeds batches"| node_training
node_regularization -->|"regularizes"| node_mlp
node_conv_math -->|"informs"| node_conv_layers
node_conv_layers -->|"composes"| node_cnn
node_attention_math -->|"generalizes to"| node_transformer_blocks
node_transformer_blocks -->|"extends to images"| node_vit
node_framework -->|"supports"| node_mnist
node_framework -->|"supports"| node_cifar
node_framework -->|"supports"| node_language_model
node_transformer_blocks -->|"assembles"| node_language_model
node_cnn -->|"forms basis"| node_cifar
node_vit -.->|"modeling option"| node_mnist

click node_scalar "https://github.com/roiamiel1/build-deep-learning-from-scratch/blob/main/stages/stage_01_scalar_values/code.py"
click node_graph "https://github.com/roiamiel1/build-deep-learning-from-scratch/blob/main/stages/stage_02_computational_graph/code.py"
click node_backprop "https://github.com/roiamiel1/build-deep-learning-from-scratch/blob/main/stages/stage_05_backprop_engine/code.py"
click node_vecmat "https://github.com/roiamiel1/build-deep-learning-from-scratch/blob/main/stages/stage_06_vector_operations/code.py"
click node_tensor "https://github.com/roiamiel1/build-deep-learning-from-scratch/blob/main/stages/stage_08_tensor_engine/code.py"
click node_layers "https://github.com/roiamiel1/build-deep-learning-from-scratch/blob/main/stages/stage_10_dense_layer/code.py"
click node_mlp "https://github.com/roiamiel1/build-deep-learning-from-scratch/blob/main/stages/stage_11_mlp/code.py"
click node_losses "https://github.com/roiamiel1/build-deep-learning-from-scratch/blob/main/stages/stage_12_loss_functions/code.py"
click node_batched_loss "https://github.com/roiamiel1/build-deep-learning-from-scratch/blob/main/stages/stage_13_loss_functions_batched/code.py"
click node_optimizers "https://github.com/roiamiel1/build-deep-learning-from-scratch/blob/main/stages/stage_14_sgd_optimizer/code.py"
click node_training "https://github.com/roiamiel1/build-deep-learning-from-scratch/blob/main/stages/stage_15_first_training_loop/code.py"
click node_batchdata "https://github.com/roiamiel1/build-deep-learning-from-scratch/blob/main/stages/stage_20_dataloader/code.py"
click node_regularization "https://github.com/roiamiel1/build-deep-learning-from-scratch/blob/main/stages/stage_21_regularization/code.py"
click node_conv_math "https://github.com/roiamiel1/build-deep-learning-from-scratch/blob/main/stages/stage_24_conv2d_math/code.py"
click node_conv_layers "https://github.com/roiamiel1/build-deep-learning-from-scratch/blob/main/stages/stage_25_conv2d_implementation/code.py"
click node_cnn "https://github.com/roiamiel1/build-deep-learning-from-scratch/blob/main/stages/stage_26_cnn_project/code.py"
click node_attention_math "https://github.com/roiamiel1/build-deep-learning-from-scratch/blob/main/stages/stage_27_attention_math/code.py"
click node_transformer_blocks "https://github.com/roiamiel1/build-deep-learning-from-scratch/blob/main/stages/stage_30_transformer/code.py"
click node_vit "https://github.com/roiamiel1/build-deep-learning-from-scratch/blob/main/stages/stage_31_vision_transformer/code.py"
click node_framework "https://github.com/roiamiel1/build-deep-learning-from-scratch/blob/main/stages/stage_32_framework_refactor/code.py"
click node_mnist "https://github.com/roiamiel1/build-deep-learning-from-scratch/blob/main/stages/stage_33_capstone_mnist/code.py"
click node_cifar "https://github.com/roiamiel1/build-deep-learning-from-scratch/blob/main/stages/stage_34_capstone_cifar10/code.py"
click node_language_model "https://github.com/roiamiel1/build-deep-learning-from-scratch/blob/main/stages/stage_35_capstone_transformer/code.py"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_scalar,node_graph,node_backprop,node_vecmat,node_tensor toneBlue
class node_layers,node_mlp,node_losses,node_batched_loss,node_optimizers,node_training,node_batchdata,node_regularization toneAmber
class node_conv_math,node_conv_layers,node_cnn toneMint
class node_attention_math,node_transformer_blocks,node_vit toneRose
class node_framework,node_mnist,node_cifar,node_language_model,node_learner toneIndigo
```

## Components

### Autodiff foundations

- [Scalar autodiff](https://github.com/roiamiel1/build-deep-learning-from-scratch/blob/main/stages/stage_01_scalar_values/code.py): `stages/stage_01_scalar_values/code.py`
- [Graph and derivatives](https://github.com/roiamiel1/build-deep-learning-from-scratch/blob/main/stages/stage_02_computational_graph/code.py): `stages/stage_02_computational_graph/code.py`
- [Backprop engine](https://github.com/roiamiel1/build-deep-learning-from-scratch/blob/main/stages/stage_05_backprop_engine/code.py): `stages/stage_05_backprop_engine/code.py`
- [Vector and matrix ops](https://github.com/roiamiel1/build-deep-learning-from-scratch/blob/main/stages/stage_06_vector_operations/code.py): `stages/stage_06_vector_operations/code.py`
- [Tensor engine](https://github.com/roiamiel1/build-deep-learning-from-scratch/blob/main/stages/stage_08_tensor_engine/code.py): `stages/stage_08_tensor_engine/code.py`

### Models and training

- [Neuron and dense layers](https://github.com/roiamiel1/build-deep-learning-from-scratch/blob/main/stages/stage_10_dense_layer/code.py): `stages/stage_10_dense_layer/code.py`
- [MLP](https://github.com/roiamiel1/build-deep-learning-from-scratch/blob/main/stages/stage_11_mlp/code.py): `stages/stage_11_mlp/code.py`
- [Loss functions](https://github.com/roiamiel1/build-deep-learning-from-scratch/blob/main/stages/stage_12_loss_functions/code.py): `stages/stage_12_loss_functions/code.py`
- [Batched losses](https://github.com/roiamiel1/build-deep-learning-from-scratch/blob/main/stages/stage_13_loss_functions_batched/code.py): `stages/stage_13_loss_functions_batched/code.py`
- [Optimizers](https://github.com/roiamiel1/build-deep-learning-from-scratch/blob/main/stages/stage_14_sgd_optimizer/code.py): `stages/stage_14_sgd_optimizer/code.py`
- [Training loop](https://github.com/roiamiel1/build-deep-learning-from-scratch/blob/main/stages/stage_15_first_training_loop/code.py): `stages/stage_15_first_training_loop/code.py`
- [Batch training and loader](https://github.com/roiamiel1/build-deep-learning-from-scratch/blob/main/stages/stage_20_dataloader/code.py): `stages/stage_20_dataloader/code.py`
- [Regularization layers](https://github.com/roiamiel1/build-deep-learning-from-scratch/blob/main/stages/stage_21_regularization/code.py): `stages/stage_21_regularization/code.py`

### Vision models

- [Convolution math](https://github.com/roiamiel1/build-deep-learning-from-scratch/blob/main/stages/stage_24_conv2d_math/code.py): `stages/stage_24_conv2d_math/code.py`
- [Convolution layers](https://github.com/roiamiel1/build-deep-learning-from-scratch/blob/main/stages/stage_25_conv2d_implementation/code.py): `stages/stage_25_conv2d_implementation/code.py`
- [CNN project](https://github.com/roiamiel1/build-deep-learning-from-scratch/blob/main/stages/stage_26_cnn_project/code.py): `stages/stage_26_cnn_project/code.py`

### Attention models

- [Attention math](https://github.com/roiamiel1/build-deep-learning-from-scratch/blob/main/stages/stage_27_attention_math/code.py): `stages/stage_27_attention_math/code.py`
- [Transformer models](https://github.com/roiamiel1/build-deep-learning-from-scratch/blob/main/stages/stage_30_transformer/code.py): `stages/stage_30_transformer/code.py`
- [Vision Transformer](https://github.com/roiamiel1/build-deep-learning-from-scratch/blob/main/stages/stage_31_vision_transformer/code.py): `stages/stage_31_vision_transformer/code.py`

### Framework and capstones

- [mytorch framework](https://github.com/roiamiel1/build-deep-learning-from-scratch/blob/main/stages/stage_32_framework_refactor/code.py): `stages/stage_32_framework_refactor/code.py`
- [MNIST capstone](https://github.com/roiamiel1/build-deep-learning-from-scratch/blob/main/stages/stage_33_capstone_mnist/code.py): `stages/stage_33_capstone_mnist/code.py`
- [CIFAR-10 capstone](https://github.com/roiamiel1/build-deep-learning-from-scratch/blob/main/stages/stage_34_capstone_cifar10/code.py): `stages/stage_34_capstone_cifar10/code.py`
- [Transformer capstone](https://github.com/roiamiel1/build-deep-learning-from-scratch/blob/main/stages/stage_35_capstone_transformer/code.py): `stages/stage_35_capstone_transformer/code.py`

### Other components

- **Learner**

## Connections

- Learner → Scalar autodiff: implements (evidence: `README.md`)
- Scalar autodiff → Graph and derivatives: extends (evidence: `README.md`)
- Graph and derivatives → Backprop engine: extends (evidence: `README.md`)
- Backprop engine → Vector and matrix ops: generalizes to (evidence: `README.md`)
- Vector and matrix ops → Tensor engine: generalizes to (evidence: `README.md`)
- Tensor engine → Neuron and dense layers: builds on (evidence: `README.md`)
- Neuron and dense layers → MLP: stacks (evidence: `README.md`)
- MLP → Loss functions: trains with (evidence: `README.md`)
- Loss functions → Batched losses: extends to batches (evidence: `README.md`)
- MLP → Optimizers: updates with (evidence: `README.md`)
- Training loop → MLP: uses (evidence: `README.md`)
- Training loop → Loss functions: computes (evidence: `README.md`)
- Training loop → Optimizers: steps (evidence: `README.md`)
- Batch training and loader → Training loop: feeds batches (evidence: `README.md`)
- Regularization layers → MLP: regularizes (evidence: `README.md`)
- Convolution math → Convolution layers: informs (evidence: `README.md`)
- Convolution layers → CNN project: composes (evidence: `README.md`)
- Attention math → Transformer models: generalizes to (evidence: `README.md`)
- Transformer models → Vision Transformer: extends to images (evidence: `README.md`)
- mytorch framework → MNIST capstone: supports (evidence: `README.md`)
- mytorch framework → CIFAR-10 capstone: supports (evidence: `README.md`)
- mytorch framework → Transformer capstone: supports (evidence: `README.md`)
- Transformer models → Transformer capstone: assembles (evidence: `README.md`)
- CNN project → CIFAR-10 capstone: forms basis (evidence: `README.md`)
- Vision Transformer → MNIST capstone: modeling option