# tinygrad/teenygrad architecture

> Architecture diagram of the GitHub repository tinygrad/teenygrad, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-03-29). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/tinygrad/teenygrad
- Repository: https://github.com/tinygrad/teenygrad

## Overview

##### Main Components
- MNIST Trainer Application: Represented by the mnist.py script that ties everything together by reading the dataset (stored under the “extra/datasets/mnist” folder) and running training.
- Frontend Interface: Highlight the key modules that form the interface for computations—tensor.py and mlops.py. These files are used by both teenygrad and tinygrad and serve as the user-facing API.
- Backend Processing: Emphasize lazy.py, which replaces tinygrad’s original backend. This module implements lazy evaluation strategies and defers execution until the operations are needed.
- Neural Network & Optimization: Include the teenygrad/nn directory (especially nn/optim.py) that handles the neural network's optimization logic.

##### Relationships and Data Flow
- Entry point (mnist.py) invoking the teenygrad frontend (mlops.py and tensor.py) to build and train the model.
- That the neural network (under teenygrad/nn) and optimization (nn/optim.py) components interact with the frontend for setting up the training loop and with the backend for performing the computations.

##### Architectural Patterns to Emphasize
- Layered Architecture: Show a clear separation of concerns between the interface (frontend), the processing engine (lazy backend, ops, realize), and the domain-specific layer (neural network and optimization logic).
- Lazy Evaluation Pattern: Clearly mark lazy.py’s role in deferring computations—making it an important element in the backend processing stack.
- Minimal Dependencies: Note that apart from numpy and tqdm, the project has no external libraries, which is crucial for understanding its lightweight and self-contained design.

## Diagram (Mermaid)

```mermaid
flowchart TD
    subgraph "Data Ingestion"
        DS["MNIST Data (extra/datasets/mnist)"]:::datasource
    end

    subgraph "Application Logic"
        MNIST["MNIST Trainer (mnist.py)"]:::app
    end

    subgraph "ML Framework Core"
        subgraph "Frontend"
            F1["Tensor API (tensor.py)"]:::frontend
            F2["MLOps API (mlops.py)"]:::frontend
        end
        subgraph "Backend"
            B["Lazy Evaluation (lazy.py)"]:::backend
            O["Operations (ops.py)"]:::backend
            R["Realization Engine (realize.py)"]:::backend
            S["Shape Inference (shape/symbolic.py)"]:::backend
            H["Helpers (helpers.py)"]:::backend
        end
        subgraph "Neural Network & Optimization"
            NN["Neural Net & Optimizers (nn/optim.py)"]:::nn
        end
    end

    subgraph "Testing & CI"
        T["Tests (test/)"]:::test
        CI["CI Workflows (.github/workflows)"]:::test
    end

    DS -->|"loads"| MNIST
    MNIST -->|"calls"| F1
    MNIST -->|"calls"| F2
    F1 -->|"uses"| B
    F2 -->|"uses"| B
    B -->|"executes"| O
    B -->|"executes"| R
    B -->|"executes"| S
    B -->|"executes"| H
    F1 -->|"trains"| NN
    F2 -->|"trains"| NN
    B -->|"computes"| NN
    MNIST -->|"tests"| T
    MNIST -->|"CItriggers"| CI

    click MNIST "https://github.com/tinygrad/teenygrad/blob/main/mnist.py"
    click F1 "https://github.com/tinygrad/teenygrad/blob/main/teenygrad/tensor.py"
    click F2 "https://github.com/tinygrad/teenygrad/blob/main/teenygrad/mlops.py"
    click B "https://github.com/tinygrad/teenygrad/blob/main/teenygrad/lazy.py"
    click O "https://github.com/tinygrad/teenygrad/blob/main/teenygrad/ops.py"
    click R "https://github.com/tinygrad/teenygrad/blob/main/teenygrad/realize.py"
    click S "https://github.com/tinygrad/teenygrad/blob/main/teenygrad/shape/symbolic.py"
    click H "https://github.com/tinygrad/teenygrad/blob/main/teenygrad/helpers.py"
    click NN "https://github.com/tinygrad/teenygrad/blob/main/teenygrad/nn/optim.py"
    click DS "https://github.com/tinygrad/teenygrad/tree/main/extra/datasets/mnist"
    click T "https://github.com/tinygrad/teenygrad/tree/main/test"
    click CI "https://github.com/tinygrad/teenygrad/blob/main/.github/workflows/test.yml"

    classDef datasource fill:#a3f3a3,stroke:#333,stroke-width:2px;
    classDef app fill:#a3c8f3,stroke:#333,stroke-width:2px;
    classDef frontend fill:#ffdead,stroke:#333,stroke-width:2px;
    classDef backend fill:#fddde6,stroke:#333,stroke-width:2px;
    classDef nn fill:#e6e6fa,stroke:#333,stroke-width:2px;
    classDef test fill:#ffbf00,stroke:#333,stroke-width:2px;
```
