# jcjohnson/pytorch-examples architecture

> Architecture diagram of the GitHub repository jcjohnson/pytorch-examples, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-05-21). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/jcjohnson/pytorch-examples
- Repository: https://github.com/jcjohnson/pytorch-examples

## Overview

##### Project type and purpose
- This is an open-source educational toolkit demonstrating how to build and train a simple two-layer fully-connected ReLU network using different levels of abstraction in PyTorch (and once in TensorFlow).
- It’s organized as a progression: raw numpy → PyTorch Tensors → PyTorch autograd → custom autograd Function → PyTorch nn → PyTorch optim → dynamic network; plus a static graph TensorFlow example for contrast.

##### File structure
- Top-level:
- build_readme.py (script to generate README), README.md / README_raw.md, LICENSE, .gitignore
- Three example directories:
– tensor/ : two_layer_net_numpy.py, two_layer_net_tensor.py
– autograd/ : two_layer_net_autograd.py, two_layer_net_custom_function.py, tf_two_layer_net.py
– nn/ : two_layer_net_nn.py, two_layer_net_optim.py, two_layer_net_module.py, dynamic_net.py
- Pattern: each directory groups examples by API level or feature set.
- There are no deployment configs or build pipelines—this is purely example code.

##### README for additional insights
- The README walks through each stage, explaining concepts (Tensors, dynamic vs static graphs, autograd, custom functions, nn Modules, optimizers, dynamic control flow).
- It describes dependencies: numpy, torch, tensorflow.

1. Data generator (random x, y)
2. Layer implementations:
- NumPy implementation (tensor/two_layer_net_numpy.py)
- PyTorch Tensor implementation (tensor/two_layer_net_tensor.py)
3. Autograd layer:
- PyTorch autograd (autograd/two_layer_net_autograd.py)
- Custom autograd Function (autograd/two_layer_net_custom_function.py)
4. Static graph comparison:
- TensorFlow static graph example (autograd/tf_two_layer_net.py)
5. High-level NN abstractions:
- nn.Sequential example (nn/two_layer_net_nn.py)
- Custom nn.Module example (nn/two_layer_net_module.py)
6. Optimizer abstraction:
- optim.Adam example (nn/two_layer_net_optim.py)
7. Dynamic control-flow example:
- DynamicNet with weight sharing (nn/dynamic_net.py)
- All components share the “Data generator” as input.
c. Architectural patterns / design principles
- Layered abstraction: low-level array ops → dynamic autodiff → high-level Modules → optimizer abstractions → dynamic control flow.
- Clear separation of concerns: data generation, computation graph definition, gradient computation, parameter update.
- Comparison of static vs dynamic graph paradigms.
d. Technologies / frameworks
- NumPy (numpy)
- PyTorch core Tensor API (torch.Tensor)
- PyTorch autograd (torch.autograd)
- PyTorch custom Function (torch.autograd.Function)
- PyTorch nn (torch.nn.Module, nn.Sequential)
- PyTorch optim (torch.optim)
- TensorFlow static graph (tf.placeholder, tf.Variable, tf.Session)

##### Detail without overcomplication
- Every major example as its own component.
- How examples group by directory / API level.
- Don’t try to model runtime loops or exact function calls; focus on component relationships and abstraction layers.

## Diagram (Mermaid)

```mermaid
flowchart TD
    %% Data Input
    Data["Data Generator"]:::data

    %% Tensor Operations Layer
    subgraph "Tensor Operations" 
        direction TB
        Numpy["two_layer_net_numpy.py"]:::external
        TorchTensor["two_layer_net_tensor.py"]:::comp
    end

    %% Autograd Layer
    subgraph "Autograd" 
        direction TB
        AutogradExample["two_layer_net_autograd.py"]:::comp
        CustomFunction["two_layer_net_custom_function.py"]:::comp
    end

    %% Static Graph Layer
    subgraph "Static Graph (TF)" 
        direction TB
        TFStatic["tf_two_layer_net.py"]:::external
    end

    %% NN Modules Layer
    subgraph "NN Modules" 
        direction TB
        Sequential["two_layer_net_nn.py"]:::comp
        CustomModule["two_layer_net_module.py"]:::comp
    end

    %% Optimizers Layer
    subgraph "Optimizers" 
        direction TB
        Optimizer["two_layer_net_optim.py"]:::optim
    end

    %% Dynamic Graph Layer
    subgraph "Dynamic Graph" 
        direction TB
        DynamicNet["dynamic_net.py"]:::comp
    end

    %% Dependencies
    Data -->|feeds| Numpy
    Data -->|feeds| TorchTensor
    Data -->|feeds| TFStatic

    TorchTensor -->|uses torch.Tensor| AutogradExample
    AutogradExample -->|calls torch.autograd.Function| CustomFunction

    AutogradExample -->|base for| Sequential
    CustomFunction -->|base for| CustomModule

    TorchTensor -->|direct support| Sequential

    Sequential -->|uses torch.optim.Adam| Optimizer
    CustomModule -->|uses torch.optim.Adam| Optimizer

    AutogradExample -->|enables| DynamicNet
    CustomModule -->|enables| DynamicNet

    %% Click Events
    click Numpy "https://github.com/jcjohnson/pytorch-examples/blob/master/tensor/two_layer_net_numpy.py"
    click TorchTensor "https://github.com/jcjohnson/pytorch-examples/blob/master/tensor/two_layer_net_tensor.py"
    click AutogradExample "https://github.com/jcjohnson/pytorch-examples/blob/master/autograd/two_layer_net_autograd.py"
    click CustomFunction "https://github.com/jcjohnson/pytorch-examples/blob/master/autograd/two_layer_net_custom_function.py"
    click TFStatic "https://github.com/jcjohnson/pytorch-examples/blob/master/autograd/tf_two_layer_net.py"
    click Sequential "https://github.com/jcjohnson/pytorch-examples/blob/master/nn/two_layer_net_nn.py"
    click CustomModule "https://github.com/jcjohnson/pytorch-examples/blob/master/nn/two_layer_net_module.py"
    click Optimizer "https://github.com/jcjohnson/pytorch-examples/blob/master/nn/two_layer_net_optim.py"
    click DynamicNet "https://github.com/jcjohnson/pytorch-examples/blob/master/nn/dynamic_net.py"

    %% Styles
    classDef data fill:#ADD8E6,stroke:#000;
    classDef comp fill:#90EE90,stroke:#000;
    classDef optim fill:#FFA500,stroke:#000;
    classDef external fill:#D3D3D3,stroke:#000;
```
