How tomiock/macrograd works
Project Type and Purpose
• Macrograd is an open-source, Python-based mini deep-learning/autograd framework.
• It provides a PyTorch-like interface with:
• A Tensor API that builds an explicit computational graph on each operation
• An execution engine that “realizes” (executes) the graph to produce data
• Automatic differentiation (backward pass)
• Model abstraction and optimizers (SGD with momentum)
• Example scripts (linear nets, MNIST) and tests
File‐Structure Analysis
• Top‐level:
• macrograd/ ← core library package
• examples/ ← demo scripts using the API
• tests/ ← unit tests
• setup.py, requirements.txt, pytest.ini, etc.
• macrograd sub‐modules:
• tensor.py ← Tensor class (data + graph hooks)
• functions.py ← definitions of ops (add, mul, exp, relu, cross_entropy…)
• graph.py ← Graph data structure (nodes, edges)
• engine.py ← graph management, default vs custom Graph factories
• backward.py ← backprop logic that walks the graph in reverse
• ast.py ← IR or AST builder for node scheduling (optional layer)
• model.py ← Model base class, layer containers, init_params
• optimizers.py ← Optimizer classes (SGD, momentum)
• examples load torchvision, tqdm, graphviz for visualization
Key Architectural Insights from README
• Inferred (lazy) execution: every Tensor op appends a node to a Graph
• Realization: calling .realize() on a Tensor/Graph triggers the engine to execute the full graph in topological order
• Backward pass: gradients are propagated node‐by‐node via backward.py
• Model & Optimizer: high‐level wrappers on top of core Tensor+Graph functionality
Main Components
a. User Layer
• Example scripts or client code (examples/*.py)
• Imports Tensor, Model, Optimizer, functions
b. API Layer
• Tensor class (creates/links graph nodes)
• Model class (defines layer sequence, init_params)
• Optimizers (SGD, Momentum)
c. Core Graph Engine
• Graph (graph.py): Node & Edge container
• AST / IR builder (ast.py) optional mapping
• Execution Engine (engine.py): realize, scheduling
• Autograd logic (backward.py): gradient propagation
d. Primitive Operations
• functions.py: mathematical & neural‐net ops (add, matmul, relu, softmax, cross_entropy)
e. Backend
• Numpy: actual numeric computation
f. Data Flow
• Data in (numpy arrays or torch tensors via examples) → wrapped by Tensor
• Tensor ops → Graph nodes → deferred computation
• Graph.realize() → Engine → Node execution → Numpy kernels → Tensor.data
• Loss computation → backward() → gradient values propagate → Optimizer.step() → parameter update
Architectural Patterns & Principles
• Lazy evaluation via explicit computational graph (similar to Theano/TensorFlow v1)
• Functional style wrappers around stateful Tensors
• Separation of concerns:
• API layer for user code
• Core engine for graph lifecycle
• Ops modules for element‐wise & matrix math
• Optimizers for parameter update logic