# chenyuxyz/teenygrad architecture

> Architecture diagram of the GitHub repository chenyuxyz/teenygrad, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-04-21). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/chenyuxyz/teenygrad
- Repository: https://github.com/chenyuxyz/teenygrad

## Overview

##### Project type and purpose
- This is a minimal deep‑learning framework + MNIST training script (“teenygrad”).
- It provides a tensor API with autograd/realtime graph compilation and a small training example (mnist.py).

##### File structure
- Top level: configuration (pre‑commit, mypy, CI), LICENSE, README
- Scripts:
- mnist.py — sample training entry point
- import_from_tinygrad.py, sz.py — utility scripts
- extra/datasets: local MNIST data files
- teenygrad/ (core library):
- tensor.py — Tensor class + frontend autograd API
- mlops.py — high‑level layers, loss functions, data utilities
- nn/optim.py — optimizer implementations (SGD, etc.)
- lazy.py — lazy evaluation graph builder (backend interface)
- ops.py — primitive tensor operations (add, matmul, etc.)
- realize.py — graph executor mapping nodes to numpy calls
- helpers.py — utility routines (e.g., random, broadcasting)
- test/: unit tests for dtype, ops, and optim
- .github/workflows/test.yml: CI runs tests on push

##### README for additional insights
- Single dependency: numpy (+ tqdm for progress bars)
- Usage: “PYTHONPATH='.' python mnist.py”
- tensor.py + mlops.py are frontends, lazy.py is the backend

a. Main components
- Training script (mnist.py)
- Dataset storage (extra/datasets/MNIST files)
- Core library packages:
- Frontend API layer: tensor.py, mlops.py
- Optimizers: nn/optim.py
- Lazy graph builder: lazy.py
- Primitive ops: ops.py
- Graph executor: realize.py
- Helpers: helpers.py
- External dependencies: numpy, tqdm
- Test suite (test/) and CI pipeline (.github/workflows)

b. Relationships and interactions
1. mnist.py loads MNIST files → constructs network via mlops & tensor APIs
2. Forward pass calls Tensor methods → lazy.py builds computational graph of ops
4. At execution, realize.py traverses the graph → issues numpy calls via helpers
5. Backward pass (autograd) is wired into Tensor → gradients flow through the same graph
6. nn/optim.py reads/writes Tensor parameters to perform updates
7. tqdm wraps loops in mnist.py for progress bars
8. test/ invokes tensor.py, ops.py, nn/optim.py directly to verify correctness
9. CI pipeline triggers test suite on every push

c. Architectural patterns & design principles
- Frontend/backend separation: user‑facing tensor/mlops vs. lazy evaluation & realizer
- Lazy evaluation & computational graph for automatic differentiation
- Modular layering: primitives (ops.py) → graph builder (lazy.py) → executor (realize.py)
- Thin dependency surface (only numpy, tqdm) for portability

d. Key technologies and libraries
- Python 3.x
- numpy (core tensor ops)
- tqdm (progress bars)
- pytest (via CI)

## Diagram (Mermaid)

```mermaid
flowchart TD
    %% User Layer
    subgraph "User Layer"
        direction TB
        Script["mnist.py"]:::script
        Data["MNIST Data"]:::script
    end

    %% Core Library - Frontend API
    subgraph "Frontend API Layer"
        direction TB
        Tensor["tensor.py (API)"]:::frontend
        MLops["mlops.py (layers)"]:::frontend
        Optim["optim.py (optimizer)"]:::frontend
    end

    %% Core Library - Backend Modules
    subgraph "Backend Modules"
        direction TB
        Lazy["lazy.py (graph builder)"]:::backend
        Ops["ops.py (primitive ops)"]:::backend
        Shape["symbolic.py (shape inference)"]:::backend
        Realize["realize.py (executor)"]:::backend
        Helpers["helpers.py (utilities)"]:::backend
    end

    %% External Dependencies & CI
    subgraph "External & CI"
        direction TB
        Numpy["numpy"]:::external
        Tqdm["tqdm"]:::external
        Tests["Test Suite"]:::external
        CI["CI Pipeline"]:::external
    end

    %% Data flow and interactions
    Script -->|"loads"| Data
    Script -->|"defines model via"| MLops
    Script -->|"uses API on"| Tensor
    Script -->|"wraps loops with"| Tqdm
    Script -->|"calls optimizer"| Optim

    MLops -->|"calls"| Tensor
    Tensor -->|"builds graph with"| Lazy
    Tensor -->|"defines ops via"| Ops
    Lazy -->|"checks shapes via"| Shape
    Lazy -->|"defines nodes via"| Ops
    Lazy -->|"outputs graph to"| Realize
    Realize -->|"executes ops on"| Numpy
    Realize -->|"uses utilities from"| Helpers
    Optim -->|"updates parameters on"| Tensor

    Tests -->|"tests"| Tensor
    Tests -->|"tests"| Ops
    Tests -->|"tests"| Optim
    CI -->|"runs"| Tests

    %% Click Events
    click Script "https://github.com/chenyuxyz/teenygrad/blob/main/mnist.py"
    click Data "https://github.com/chenyuxyz/teenygrad/tree/main/extra/datasets/mnist/"
    click Tensor "https://github.com/chenyuxyz/teenygrad/blob/main/teenygrad/tensor.py"
    click MLops "https://github.com/chenyuxyz/teenygrad/blob/main/teenygrad/mlops.py"
    click Optim "https://github.com/chenyuxyz/teenygrad/blob/main/teenygrad/nn/optim.py"
    click Lazy "https://github.com/chenyuxyz/teenygrad/blob/main/teenygrad/lazy.py"
    click Ops "https://github.com/chenyuxyz/teenygrad/blob/main/teenygrad/ops.py"
    click Shape "https://github.com/chenyuxyz/teenygrad/blob/main/teenygrad/shape/symbolic.py"
    click Realize "https://github.com/chenyuxyz/teenygrad/blob/main/teenygrad/realize.py"
    click Helpers "https://github.com/chenyuxyz/teenygrad/blob/main/teenygrad/helpers.py"
    click Tests "https://github.com/chenyuxyz/teenygrad/tree/main/test/"
    click CI "https://github.com/chenyuxyz/teenygrad/blob/main/.github/workflows/test.yml"

    %% Styles
    classDef script fill:#e0ffe0,stroke:#2d862d;
    classDef frontend fill:#e0f0ff,stroke:#1f4383;
    classDef backend fill:#e6f7ff,stroke:#1f569d;
    classDef external fill:#ffe0b3,stroke:#b36200;
```
