# chenyuxyz/tinygrad architecture

> Architecture diagram of the GitHub repository chenyuxyz/tinygrad, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-04-21). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/chenyuxyz/tinygrad
- Repository: https://github.com/chenyuxyz/tinygrad

## Overview

##### Project type and purpose
- tinygrad is a lightweight deep‑learning framework, akin to “RISC” versus XLA’s “CISC.” It provides:
- A Python API (Tensor, nn, optim) for model definition and training
- A lazy autograd/kernel fusion engine
- A modular code‑generation pipeline
- A pluggable runtime for multiple hardware backends (CPU, GPU, WebGPU, CUDA, Metal, etc.)
- Its goal is extreme simplicity and ease of adding new accelerators, while still supporting inference and training on real models (LLaMA, Stable Diffusion).

##### File structure
- Top‐level directories:
- tinygrad/             ← core library
- docs/                 ← documentation sources, MkDocs config (mkdocs.yml)
- examples/             ← runnable examples showcasing usage & benchmarks
- extra/                ← experimental drivers, tooling, proofs‐of‐concept
- test/                 ← end‑to‑end, external, unit, fuzz tests
- .github/              ← CI workflows, custom GitHub Actions
- Core library (tinygrad/) layers:
- frontend/             ← ONNX/Torch import
- tensor.py + dtype.py  ← Tensor API + data types
- gradient.py           ← Autograd logic
- codegen/              ← kernel fusion, lowering, symbolic transforms
- renderer/             ← IR → C/LLVM/PTX/WGSL code emitters
- runtime/              ← autogen bindings + graph execution + ops implementations
- engine/               ← scheduling, memory planning, JIT hooks
- nn/                   ← high‐level neural network layers, datasets, optimizers
- viz/                  ← graph and profiling visualization

##### README for additional insights
- Confirms multi‑accelerator support (~25 low‑level ops per device)
- Emphasizes “laziness” (lazy graph construction + fusion)
- Points to examples & docs, as well as CI tests and process‑replay for kernel stability

##### Detail
- Every core subfolder in tinygrad/: codegen, renderer, runtime, engine, frontend, nn
- Autogen directory under runtime as a dynamic binding factory
- How examples/tests invoke the pipeline for benchmarking and validation
- CI workflows that run tests, docs builds, benchmarks

## Diagram (Mermaid)

```mermaid
flowchart TD
    %% Core Library Modules
    subgraph "User API Layer"
        Tensor["Tensor"]:::core
        DType["Data Types"]:::core
        Grad["Autograd"]:::core
        NN_Optim["Optimizers"]:::core
        NN_Data["Datasets"]:::core
        NN_State["State"]:::core
    end

    subgraph "Frontend Parsers"
        FE_ONNX["ONNX Parser"]:::core
        FE_Torch["TorchScript Parser"]:::core
    end

    subgraph "Computation Graph & Lazy Engine"
        Engine_Init["Engine Init"]:::core
        Grouper["Grouper"]:::core
        JIT["JIT Hooks"]:::core
        Memory["Memory Planner"]:::core
        Schedule["Scheduler"]:::core
        Search["Search"]:::core
        Realize["Realize"]:::core
    end

    subgraph "Codegen & Fusion Pipeline"
        CG_Init["Codegen Init"]:::core
        Devectorizer["Devectorizer"]:::core
        Expander["Expander"]:::core
        Heuristic["Heuristic Fusion"]:::core
        Kernel["Kernel Builder"]:::core
        Linearize["Linearizer"]:::core
        Lowerer["Lowerer"]:::core
        Symbolic["Symbolic"]:::core
        Transcendental["Transcendental"]:::core
    end

    subgraph "Renderer / Backend IR Emitters"
        REN_Init["Renderer Init"]:::core
        CStyle["C Emitter"]:::core
        LLVMIR["LLVM IR Emitter"]:::core
        PTX["PTX Emitter"]:::core
        WGSL["WGSL Emitter"]:::core
    end

    subgraph "Runtime Dispatch & Autogen Bindings"
        RT_Init["Runtime Init"]:::core
        Autogen["Autogen Bindings"]:::core
        GraphExec["Graph Executors"]:::core
        Ops_CPU["CPU Ops"]:::core
        Ops_CUDA["CUDA Ops"]:::core
        Ops_HIP["HIP Ops"]:::core
        Ops_Metal["Metal Ops"]:::core
        Ops_WebGPU["WebGPU Ops"]:::core
        Allocator["Allocator"]:::core
        Comp_CUDA["CUDA Compiler"]:::core
        Comp_AMD["AMD Compiler"]:::core
    end

    subgraph "Backends & Hardware"
        DRIVERS["Device Drivers"]:::external
        HW["Hardware Accelerators"]:::external
    end

    subgraph "Tools & Pipelines"
        DOCS["Documentation (MkDocs)"]:::tools
        EXAMPLES["Examples & Benchmarks"]:::tools
        CI["CI Workflows"]:::tools
        TESTS["Tests"]:::tools
        VIZ["Visualization & Profiling"]:::tools
    end

    %% Data Flow
    Tensor -->|imports ops| Grad
    Tensor -->|uses| DType
    Grad -->|builds graph| Engine_Init
    NN_Optim -->|uses autograd| Grad
    NN_Data -->|data loaders| NN_Optim
    NN_State -->|manages state| NN_Optim

    FE_ONNX -->|parses model| Engine_Init
    FE_Torch -->|parses model| Engine_Init

    Engine_Init -->|groups ops| Grouper
    Grouper -->|jit hooks| JIT
    JIT -->|plan memory| Memory
    Memory -->|schedule| Schedule
    Schedule -->|search| Search
    Search -->|realize graph| Realize
    Realize -->|generates uop graph| CG_Init

    CG_Init -->|devectorize| Devectorizer
    Devectorizer --> Expander
    Expander --> Heuristic
    Heuristic --> Kernel
    Kernel --> Linearize
    Linearize --> Lowerer
    Lowerer --> Symbolic
    Symbolic --> Transcendental
    Transcendental --> REN_Init

    REN_Init --> CStyle
    REN_Init --> LLVMIR
    REN_Init --> PTX
    REN_Init --> WGSL

    CStyle -->|emit C| RT_Init
    LLVMIR -->|emit LLVM| RT_Init
    PTX -->|emit PTX| RT_Init
    WGSL -->|emit WGSL| RT_Init

    RT_Init --> Autogen
    Autogen --> GraphExec
    GraphExec --> Ops_CPU
    GraphExec --> Ops_CUDA
    GraphExec --> Ops_HIP
    GraphExec --> Ops_Metal
    GraphExec --> Ops_WebGPU
    Ops_CPU -->|dispatch| DRIVERS
    Ops_CUDA --> DRIVERS
    Ops_HIP --> DRIVERS
    Ops_Metal --> DRIVERS
    Ops_WebGPU --> DRIVERS
    DRIVERS -->|execute| HW
    Allocator --> RT_Init
    Comp_CUDA --> RT_Init
    Comp_AMD --> RT_Init

    DOCS -.->|build| MKDOCS[mkdocs.yml]
    EXAMPLES -.->|run pipeline| Engine_Init
    CI -.->|test workflows| TESTS
    TESTS -.->|validate| Engine_Init
    VIZ -.->|visualize| Engine_Init

    %% Click Events
    click Tensor "https://github.com/chenyuxyz/tinygrad/blob/master/tinygrad/tensor.py"
    click DType "https://github.com/chenyuxyz/tinygrad/blob/master/tinygrad/dtype.py"
    click Grad "https://github.com/chenyuxyz/tinygrad/blob/master/tinygrad/gradient.py"
    click NN_Optim "https://github.com/chenyuxyz/tinygrad/blob/master/tinygrad/nn/optim.py"
    click NN_Data "https://github.com/chenyuxyz/tinygrad/blob/master/tinygrad/nn/datasets.py"
    click NN_State "https://github.com/chenyuxyz/tinygrad/blob/master/tinygrad/nn/state.py"
    click FE_ONNX "https://github.com/chenyuxyz/tinygrad/blob/master/tinygrad/frontend/onnx.py"
    click FE_Torch "https://github.com/chenyuxyz/tinygrad/blob/master/tinygrad/frontend/torch.py"
    click Engine_Init "https://github.com/chenyuxyz/tinygrad/blob/master/tinygrad/engine/__init__.py"
    click Grouper "https://github.com/chenyuxyz/tinygrad/blob/master/tinygrad/engine/grouper.py"
    click JIT "https://github.com/chenyuxyz/tinygrad/blob/master/tinygrad/engine/jit.py"
    click Memory "https://github.com/chenyuxyz/tinygrad/blob/master/tinygrad/engine/memory.py"
    click Schedule "https://github.com/chenyuxyz/tinygrad/blob/master/tinygrad/engine/schedule.py"
    click Search "https://github.com/chenyuxyz/tinygrad/blob/master/tinygrad/engine/search.py"
    click Realize "https://github.com/chenyuxyz/tinygrad/blob/master/tinygrad/engine/realize.py"
    click CG_Init "https://github.com/chenyuxyz/tinygrad/blob/master/tinygrad/codegen/__init__.py"
    click Devectorizer "https://github.com/chenyuxyz/tinygrad/blob/master/tinygrad/codegen/devectorizer.py"
    click Expander "https://github.com/chenyuxyz/tinygrad/blob/master/tinygrad/codegen/expander.py"
    click Heuristic "https://github.com/chenyuxyz/tinygrad/blob/master/tinygrad/codegen/heuristic.py"
    click Kernel "https://github.com/chenyuxyz/tinygrad/blob/master/tinygrad/codegen/kernel.py"
    click Linearize "https://github.com/chenyuxyz/tinygrad/blob/master/tinygrad/codegen/linearize.py"
    click Lowerer "https://github.com/chenyuxyz/tinygrad/blob/master/tinygrad/codegen/lowerer.py"
    click Symbolic "https://github.com/chenyuxyz/tinygrad/blob/master/tinygrad/codegen/symbolic.py"
    click Transcendental "https://github.com/chenyuxyz/tinygrad/blob/master/tinygrad/codegen/transcendental.py"
    click REN_Init "https://github.com/chenyuxyz/tinygrad/blob/master/tinygrad/renderer/__init__.py"
    click CStyle "https://github.com/chenyuxyz/tinygrad/blob/master/tinygrad/renderer/cstyle.py"
    click LLVMIR "https://github.com/chenyuxyz/tinygrad/blob/master/tinygrad/renderer/llvmir.py"
    click PTX "https://github.com/chenyuxyz/tinygrad/blob/master/tinygrad/renderer/ptx.py"
    click WGSL "https://github.com/chenyuxyz/tinygrad/blob/master/tinygrad/renderer/wgsl.py"
    click RT_Init "https://github.com/chenyuxyz/tinygrad/blob/master/tinygrad/runtime/__init__.py"
    click Autogen "https://github.com/chenyuxyz/tinygrad/tree/master/tinygrad/runtime/autogen/"
    click GraphExec "https://github.com/chenyuxyz/tinygrad/tree/master/tinygrad/runtime/graph/"
    click Ops_CPU "https://github.com/chenyuxyz/tinygrad/blob/master/tinygrad/runtime/ops_cpu.py"
    click Ops_CUDA "https://github.com/chenyuxyz/tinygrad/blob/master/tinygrad/runtime/ops_cuda.py"
    click Ops_HIP "https://github.com/chenyuxyz/tinygrad/blob/master/tinygrad/runtime/ops_hip.py"
    click Ops_Metal "https://github.com/chenyuxyz/tinygrad/blob/master/tinygrad/runtime/ops_metal.py"
    click Ops_WebGPU "https://github.com/chenyuxyz/tinygrad/blob/master/tinygrad/runtime/ops_webgpu.py"
    click Allocator "https://github.com/chenyuxyz/tinygrad/blob/master/tinygrad/runtime/support/allocator.py"
    click Comp_CUDA "https://github.com/chenyuxyz/tinygrad/blob/master/tinygrad/runtime/support/compiler_cuda.py"
    click Comp_AMD "https://github.com/chenyuxyz/tinygrad/blob/master/tinygrad/runtime/support/compiler_amd.py"
    click DOCS "https://github.com/chenyuxyz/tinygrad/blob/master/mkdocs.yml"
    click EXAMPLES "https://github.com/chenyuxyz/tinygrad/tree/master/examples/"
    click CI "https://github.com/chenyuxyz/tinygrad/blob/master/.github/workflows/test.yml"
    click TESTS "https://github.com/chenyuxyz/tinygrad/tree/master/test/"
    click VIZ "https://github.com/chenyuxyz/tinygrad/blob/master/tinygrad/viz/serve.py"

    %% Styles
    classDef core fill:#cce5ff,stroke:#333,stroke-width:1px;
    classDef external fill:#d4edda,stroke:#333,stroke-width:1px;
    classDef tools fill:#f0f0f0,stroke:#333,stroke-width:1px;
```
