How mpezeshki/pytorch_forward_forward works
Step 1: Identify Project Type and Purpose
• This is a single-script PyTorch implementation of Geoffrey Hinton’s Forward-Forward (FF) training algorithm, intended for MNIST digit classification as an alternative to back-propagation.
• The project contains one entry point (main.py), an image folder for documentation (imgs/), and standard metadata files (.gitignore, LICENSE, README.md).
Step 2: Analyze the File Structure
• main.py: orchestrates data loading, model definition, local forward-forward training loops, and evaluation.
• README.md: explains algorithm, sample generation, local objectives, and usage.
Step 3: Examine README for Insights
• Core components described:
• Positive and negative sample merge (image+label vs. image+random_label)
• Local threshold-based objective at each layer
• Sequential training of layers without backward pass
• Final prediction by maximizing total layer activations
• Dependencies: Python, PyTorch, torchvision (for MNIST).
a. Main Components:
1. Data Loader
2. Positive Sample Generator
3. Negative Sample Generator
4. Layer Modules (Layer 1, Layer 2, …, Layer N)
5. Local Training Objectives (per layer)
6. Training Orchestrator (main loop)
7. Prediction Engine
8. External Frameworks (PyTorch, torchvision)
b. Relationships & Interactions:
• Data Loader → Positive/Negative Sample Generators
• Each Generator → Feed inputs to each Layer Module
• Layer Module ↔ Local Training Objective (computes local loss)
• Training Orchestrator sequences Layer-wise training
• After training, Prediction Engine queries all y ∈ [0..9]; sends (x,y) through Layers → selects y with max total activation.
c. Architectural Patterns:
• Pipeline of self-contained “learning blocks” (layers) each with own local objective.
• Single-process, sequential orchestration (no distributed or microservice architecture).
d. Key Technologies:
• Python interpreter
• PyTorch nn.Module for layer definitions
• torchvision.datasets.MNIST
• CLI entry point in main.py
• That “frontend” is the CLI/user invoking main.py, “backend” is the PyTorch pipeline.
• No separate UI or web layer; it’s a batch training application.
• How images in imgs/ document architecture but are not part of runtime.
1. Rectangles for each component (Data Loader, Sample Generators, Layer Modules, Training Orchestrator, Prediction Engine).
• From Data Loader → Sample Generators
• From Generators → Layer Modules
• From Layer Modules → Local Objectives
• From Layer Modules → Next Layer (during forward pass)
• From Prediction Engine → Output decision