# tomiock/macrograd architecture

> Architecture diagram of the GitHub repository tomiock/macrograd, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-05-30). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/tomiock/macrograd
- Repository: https://github.com/tomiock/macrograd

## Overview

##### Project Type and Purpose
- Macrograd is an open-source, Python-based mini deep-learning/autograd framework.
- It provides a PyTorch-like interface with:
- A Tensor API that builds an explicit computational graph on each operation
- An execution engine that “realizes” (executes) the graph to produce data
- Automatic differentiation (backward pass)
- Model abstraction and optimizers (SGD with momentum)
- Example scripts (linear nets, MNIST) and tests

##### File‐Structure Analysis
- Top‐level:
- macrograd/      ← core library package
- examples/       ← demo scripts using the API
- tests/          ← unit tests
- setup.py, requirements.txt, pytest.ini, etc.
- macrograd sub‐modules:
- tensor.py       ← Tensor class (data + graph hooks)
- functions.py    ← definitions of ops (add, mul, exp, relu, cross_entropy…)
- graph.py        ← Graph data structure (nodes, edges)
- engine.py       ← graph management, default vs custom Graph factories
- backward.py     ← backprop logic that walks the graph in reverse
- ast.py          ← IR or AST builder for node scheduling (optional layer)
- model.py        ← Model base class, layer containers, init_params
- optimizers.py   ← Optimizer classes (SGD, momentum)
- examples load torchvision, tqdm, graphviz for visualization

##### Key Architectural Insights from README
- Inferred (lazy) execution: every Tensor op appends a node to a Graph
- Realization: calling .realize() on a Tensor/Graph triggers the engine to execute the full graph in topological order
- Backward pass: gradients are propagated node‐by‐node via backward.py
- Model & Optimizer: high‐level wrappers on top of core Tensor+Graph functionality

##### Main Components
a. User Layer
- Example scripts or client code (examples/*.py)
- Imports Tensor, Model, Optimizer, functions
b. API Layer
- Tensor class (creates/links graph nodes)
- Model class (defines layer sequence, init_params)
- Optimizers (SGD, Momentum)
c. Core Graph Engine
- Graph (graph.py): Node & Edge container
- AST / IR builder (ast.py) optional mapping
- Execution Engine (engine.py): realize, scheduling
- Autograd logic (backward.py): gradient propagation
d. Primitive Operations
- functions.py: mathematical & neural‐net ops (add, matmul, relu, softmax, cross_entropy)
e. Backend
- Numpy: actual numeric computation
f. Data Flow
- Data in (numpy arrays or torch tensors via examples) → wrapped by Tensor
- Tensor ops → Graph nodes → deferred computation
- Graph.realize() → Engine → Node execution → Numpy kernels → Tensor.data
- Loss computation → backward() → gradient values propagate → Optimizer.step() → parameter update

##### Architectural Patterns & Principles
- Lazy evaluation via explicit computational graph (similar to Theano/TensorFlow v1)
- Functional style wrappers around stateful Tensors
- Separation of concerns:
- API layer for user code
- Core engine for graph lifecycle
- Ops modules for element‐wise & matrix math
- Optimizers for parameter update logic

## Diagram (Mermaid)

```mermaid
flowchart TB
    %% User Layer
    subgraph "User Layer"
        Examples["Example Scripts\nexamples/*.py"]:::user
        Tests["Unit Tests\ntests/test_main.py"]:::user
    end

    %% API Layer
    subgraph "API Layer"
        Tensor["Tensor\nwraps numpy array, attaches graph"]:::api
        Model["Model\nlayer container, init_params"]:::api
        Optimizer["Optimizers\nSGD, momentum"]:::api
    end

    %% Core Graph Engine
    subgraph "Core Graph Engine"
        Graph["Graph\nstores nodes/edges"]:::engine
        AST["AST/IR Builder\nnode scheduling"]:::engine
        Engine["Execution Engine\nrealize, scheduling"]:::engine
        Backward["Autodiff Logic\nreverse-mode"]:::engine
    end

    %% Ops & Utils
    subgraph "Ops & Utils"
        Functions["Functions\nadd, mul, relu, softmax, cross_entropy"]:::ops
        Utils["Utils\nshape broadcasting & inference"]:::ops
    end

    %% Backend
    subgraph "Backend"
        Numpy["NumPy"]:::backend
        Graphviz["Graphviz\nvisualization"]:::backend
    end

    %% Connections
    Examples -->|"import/use"| Tensor
    Examples -->|"import"| Model
    Examples -->|"import"| Optimizer
    Tests -->|"import/use"| Tensor
    Tests -->|"import"| Model
    Tests -->|"import"| Optimizer

    Tensor -->|"create node"| Graph
    Tensor -->|".realize()"| Engine
    Model -->|"defines layers"| Tensor
    Model -->|"builds graph"| Graph
    Functions -->|"used by"| Tensor
    Utils -->|"used by"| Tensor
    Utils -->|"used by"| Functions

    Graph -->|"schedule nodes"| AST
    AST -->|"compile graph"| Engine
    Engine -->|"execute node"| Numpy
    Engine -->|"visualize graph"| Graphviz
    Engine -->|"invoke backward"| Backward
    Backward -->|"compute gradients"| Engine
    Backward -->|"gradients"| Optimizer
    Optimizer -->|"update params"| Tensor

    %% Click Events
    click Tensor "https://github.com/tomiock/macrograd/blob/main/macrograd/tensor.py"
    click Functions "https://github.com/tomiock/macrograd/blob/main/macrograd/functions.py"
    click Utils "https://github.com/tomiock/macrograd/blob/main/macrograd/utils_shape.py"
    click Graph "https://github.com/tomiock/macrograd/blob/main/macrograd/graph.py"
    click AST "https://github.com/tomiock/macrograd/blob/main/macrograd/ast.py"
    click Engine "https://github.com/tomiock/macrograd/blob/main/macrograd/engine.py"
    click Backward "https://github.com/tomiock/macrograd/blob/main/macrograd/backward.py"
    click Model "https://github.com/tomiock/macrograd/blob/main/macrograd/model.py"
    click Optimizer "https://github.com/tomiock/macrograd/blob/main/macrograd/optimizers.py"
    click Examples "https://github.com/tomiock/macrograd/blob/main/examples/linear_net.py"
    click Tests "https://github.com/tomiock/macrograd/blob/main/tests/test_main.py"

    %% Styles
    classDef user fill:#ccf,stroke:#333,stroke-width:1px
    classDef api fill:#ccf,stroke:#333,stroke-width:1px
    classDef engine fill:#cfc,stroke:#333,stroke-width:1px
    classDef ops fill:#fc9,stroke:#333,stroke-width:1px
    classDef backend fill:#ddd,stroke:#333,stroke-width:1px
```
