How jcjohnson/pytorch-examples works
Project type and purpose
• This is an open-source educational toolkit demonstrating how to build and train a simple two-layer fully-connected ReLU network using different levels of abstraction in PyTorch (and once in TensorFlow).
• It’s organized as a progression: raw numpy → PyTorch Tensors → PyTorch autograd → custom autograd Function → PyTorch nn → PyTorch optim → dynamic network; plus a static graph TensorFlow example for contrast.
File structure
• Top-level:
• build_readme.py (script to generate README), README.md / README_raw.md, LICENSE, .gitignore
• Three example directories:
– tensor/ : two_layer_net_numpy.py, two_layer_net_tensor.py
– autograd/ : two_layer_net_autograd.py, two_layer_net_custom_function.py, tf_two_layer_net.py
– nn/ : two_layer_net_nn.py, two_layer_net_optim.py, two_layer_net_module.py, dynamic_net.py
• Pattern: each directory groups examples by API level or feature set.
• There are no deployment configs or build pipelines—this is purely example code.
README for additional insights
• The README walks through each stage, explaining concepts (Tensors, dynamic vs static graphs, autograd, custom functions, nn Modules, optimizers, dynamic control flow).
• It describes dependencies: numpy, torch, tensorflow.
1. Data generator (random x, y)
2. Layer implementations:
• NumPy implementation (tensor/two_layer_net_numpy.py)
• PyTorch Tensor implementation (tensor/two_layer_net_tensor.py)
3. Autograd layer:
• PyTorch autograd (autograd/two_layer_net_autograd.py)
• Custom autograd Function (autograd/two_layer_net_custom_function.py)
4. Static graph comparison:
• TensorFlow static graph example (autograd/tf_two_layer_net.py)
5. High-level NN abstractions:
• nn.Sequential example (nn/two_layer_net_nn.py)
• Custom nn.Module example (nn/two_layer_net_module.py)
6. Optimizer abstraction:
• optim.Adam example (nn/two_layer_net_optim.py)
7. Dynamic control-flow example:
• DynamicNet with weight sharing (nn/dynamic_net.py)
• All components share the “Data generator” as input.
c. Architectural patterns / design principles
• Layered abstraction: low-level array ops → dynamic autodiff → high-level Modules → optimizer abstractions → dynamic control flow.
• Clear separation of concerns: data generation, computation graph definition, gradient computation, parameter update.
• Comparison of static vs dynamic graph paradigms.
d. Technologies / frameworks
• NumPy (numpy)
• PyTorch core Tensor API (torch.Tensor)
• PyTorch autograd (torch.autograd)
• PyTorch custom Function (torch.autograd.Function)
• PyTorch nn (torch.nn.Module, nn.Sequential)
• PyTorch optim (torch.optim)
• TensorFlow static graph (tf.placeholder, tf.Variable, tf.Session)
Detail without overcomplication
• Every major example as its own component.
• How examples group by directory / API level.
• Don’t try to model runtime loops or exact function calls; focus on component relationships and abstraction layers.