How rodsiry/nem works
PROJECT OVERVIEW
• Purpose: The project implements the NEM algorithm which evolves neuron update rules (via genetic search) to achieve efficient continual learning on class-incremental tasks (with MNIST as a primary example, and evaluations on tasks like SVHN).
• Type: This is a research/experimental, single-process script-based project designed primarily for algorithm development, training, testing, and visualizing learned neural filters.
ANALYZING THE FILE STRUCTURE
• README.md: Offers an overview of the project, instructions to run the main components, and a reference to the original research paper.
• nem.py: Likely the main entry point for launching the genetic search or the meta-training procedure.
• nem_test.py: The test harness for evaluating trained update rules on various meta-test tasks.
• basic_train.py: Presumably provides an alternative or base training loop for running experiments.
• models.py: Contains the neural network model definitions or the specific implementation of NEM’s neuron model.
• loaders.py: Holds data loading routines, ensuring that tasks like MNIST and SVHN are correctly fed into the system.
• plot.py: Provides plotting and visualization capabilities to display the learned filters.
• utils.py: A collection of helper functions probably used across modules (e.g., logging, configuration, metrics).
A. Core Components:
• Genetic Search / Meta-Training Module (nem.py):
• Central executable for launching the genetic optimization process.
• Implements the search for update rules that exhibit continual learning.
• Test/Evaluation Module (nem_test.py):
• Evaluates the learned update rules on meta-test datasets like MNIST and SVHN.
• Triggers filter visualization via the plot module.
• Training Module (basic_train.py):
• Provides baseline or alternative training routines.
B. Supporting Components:
• Data Loading Layer (loaders.py):
• Manages dataset import and preprocessing.
• Ensures proper data flow from raw input to the training/evaluation modules.
• Model Definitions (models.py):
• Contains model architectures and possibly the NEM neuron implementation.
• Visualization (plot.py):
• Handles displaying the learned filters and other experiment results.
• Utility Functions (utils.py):
• Provides shared functionality (e.g., logging, configuration, error handling) across modules.
CONNECTIONS AND INTERACTIONS
• Data Flow:
• Data starts at the Data Loading Layer, feeding into both the training (basic_train.py) and testing (nem_test.py) pipelines.
• The output from the training modules and the evolution process in nem.py produces updated models and update rules which will later be tested.
• Module Dependencies:
• “nem.py” orchestrates the meta-training genetic search process and interfaces with “models.py” for neural computations.
• “nem_test.py” depends on the outputs of “nem.py” and “basic_train.py” to evaluate performance and invoke visualizations via “plot.py.”
• “utils.py” is integrated across the system to support logging, experiments configuration, and utility operations.
ARCHITECTURAL PRINCIPLES AND TECHNOLOGIES
• Design Pattern: The structure can conceptually follow a modular design where each file represents a self-contained component:
• Model Layer (models.py)
• Data Layer (loaders.py)
• Training/Evaluation Layer (nem.py, basic_train.py, nem_test.py)
• Visualization and Utility Support (plot.py, utils.py)
• Experimental Workflow: Emphasize the separation of experimentation phases – genetic search/meta-training vs. evaluation and visualization.
• Technologies: Although not explicitly detailed in the README, expect Python’s scientific ecosystem (e.g., NumPy, potentially PyTorch or TensorFlow if used in models) to be underlying, along with libraries for plotting.
FINAL REMARKS FOR THE PRINCIPAL SOFTWARE ENGINEER
• Being as detailed as possible. Do not shy away from separating the components into fine-grained modules even if that results in many parts. This ensures that all architectural elements, especially those critical for understanding the experimental workflow (meta-training, evaluation, and visualization), are captured.
• Confirm that every module mentioned in the file tree is accounted for and their relationships are clearly illustrated.