# neuromorphiccomputing/softhebb architecture

> Architecture diagram of the GitHub repository neuromorphiccomputing/softhebb, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-09-16). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/neuromorphiccomputing/softhebb
- Repository: https://github.com/neuromorphiccomputing/softhebb

## Overview

##### Project Type and Purpose
- This is an open-source Python codebase for reproducing and extending the SoftHebb unsupervised Hebbian deep-learning algorithm (ICLR 2023 / Neuromorphic Comput. Eng. 2022).

##### Main Components
A. Entry-point Scripts
- demo.py: quick start for training and evaluation
- train.py / multi_layer.py: orchestrate training of one model across epochs
- ray_search.py: hyperparameter optimization harness using Ray Tune
- post_hoc_loss.py: compute post-hoc loss analyses
- graphs.ipynb: generate plots and figures
B. Configuration
- presets.json (or preset.json): defines network architectures, datasets, layer hyperparameters
- environment_pytorch==1.7.1.yml: conda env specification
C. Data and Dataset Management
- dataset.py: interfaces to download / load MNIST, CIFAR, ImageNette, ImageNet
- utils.py: global constants (DATA path), shared helper functions
D. Model Definition
- layer.py, hebblinear.py, hebbconv.py, multi_layer.py (model wrapper)
- activation.py: custom activation layers (soft winner-take-all, RePU, etc.)
- model.py: load/save checkpoint and define full multilayer models
E. Training Engine
- engine.py: one-epoch train/eval logic for a single model
- train.py: multi-epoch training loops for different modes (unsup, hybrid, supervised)
- log.py: logging facility for metrics and convergence stats
F. Hyperparameter Search
- ray_search.py: defines search space, runner logic, TensorBoard output
G. Results & Outputs
- DATA/hebb/results: models, per-layer checkpoints, network checkpoints
- DATA/hebb/search: Ray Tune logs for visualization

##### Data Flow & Interactions
1. User invokes a script (demo.py, multi_layer.py or ray_search.py) from CLI
2. Script reads preset.json to build a sequence of blocks:
- Blocks = {Flatten, BatchNorm, Convolutional or Fully-connected + Activation + Pooling}
3. dataset.py + utils.py load and preprocess data from DATA/dataset/…
4. model.py / multi_layer.py instantiate the PyTorch model according to presets
5. engine.py applies per-epoch training:
- For Hebbian layers: local weight updates (hebblinear/hebbconv rules)
- Optionally supervised BP on classifier block
6. log.py captures and persists metrics and layer statistics to disk
7. ray_search.py spins up multiple trials in parallel via Ray, writes TensorBoard logs under DATA/hebb/search
8. After training, graphs.ipynb / nb_utils.py produce figures for ICLR / paper

##### Architectural Patterns & Design Principles
- Modular “block” pattern: each network is a pipeline of interchangeable blocks (MLP or CNN blocks) configured purely via JSON presets.
- Separation of concerns:
- Data loading (dataset.py)
- Model definition (layer.py, model.py)
- Training logic (engine.py, train.py)
- Orchestration / CLI (multi_layer.py, ray_search.py)
- Visualization (graphs.ipynb)

##### Final Recommendations
- Every major directory/file grouping as a logical subsystem.
- Pipeline nature: presets.json drives everything.
- Verify that connections represent the actual Python imports and data dependencies.
- Keep it detailed but uncluttered: group low-level functions (utils, activations) under “Utilities” if needed.

## Diagram (Mermaid)

```mermaid
flowchart TB
    %% Entry-Point Scripts
    subgraph "Entry-Point Scripts"
        direction TB
        Demo["demo.py"]:::scripts
        TrainCLI["train.py"]:::scripts
        MultiLayerCLI["multi_layer.py"]:::scripts
        RaySearchCLI["ray_search.py"]:::scripts
        PostHoc["post_hoc_loss.py"]:::scripts
        GraphsNotebook["graphs.ipynb"]:::scripts
    end

    %% Configuration
    subgraph "Configuration"
        direction TB
        Presets["presets.json"]:::config
        CondaEnv["environment_pytorch==1.7.1.yml"]:::config
    end

    %% Data & Dataset Management
    subgraph "Data & Dataset Management"
        direction TB
        Dataset["dataset.py"]:::data
        Utils["utils.py"]:::data
    end

    %% Model Definition
    subgraph "Model Definition"
        direction TB
        LayerBlocks["layer.py"]:::model
        HebbLinear["hebblinear.py"]:::model
        HebbConv["hebbconv.py"]:::model
        ActivationFuncs["activation.py"]:::model
        ModelWrapper["model.py"]:::model
    end

    %% Training Engine
    subgraph "Training Engine"
        direction TB
        OneEpoch["engine.py"]:::training
        MultiEpoch["train.py"]:::training
        Logger["log.py"]:::training
    end

    %% Hyperparameter Search
    subgraph "Hyperparameter Search"
        direction TB
        RaySearch["ray_search.py"]:::search
        ParallelTrials["Parallel Trials"]:::search
        TensorBoard["TensorBoard Logs"]:::search
    end

    %% Visualization & Notebooks
    subgraph "Visualization & Notebooks"
        direction TB
        NBUtils["nb_utils.py"]:::vis
        GraphsNotebook
    end

    %% Results Stores
    Results["DATA/hebb/results"]:::datastore
    SearchLogs["DATA/hebb/search"]:::datastore
    Images["images/"]:::datastore

    %% Dependencies and Data Flow
    Demo -->|reads| Presets
    TrainCLI -->|reads| Presets
    MultiLayerCLI -->|reads| Presets
    RaySearchCLI -->|reads| Presets
    PostHoc -->|reads results| Results
    GraphsNotebook -->|loads| Results

    Presets --> ModelWrapper
    Presets --> LayerBlocks

    Dataset -->|provides data| ModelWrapper
    Utils -->|helpers| Dataset

    ModelWrapper --> OneEpoch
    ModelWrapper --> MultiEpoch
    LayerBlocks --> OneEpoch
    HebbLinear --> OneEpoch
    HebbConv --> OneEpoch
    ActivationFuncs --> OneEpoch

    OneEpoch -->|epoch results| Logger
    MultiEpoch -->|loop| OneEpoch
    Logger --> Results

    RaySearchCLI --> RaySearch
    RaySearch --> ParallelTrials
    ParallelTrials --> OneEpoch
    RaySearch --> TensorBoard
    TensorBoard --> SearchLogs

    GraphsNotebook --> NBUtils
    NBUtils --> Results
    Images --> GraphsNotebook

    %% Click Events
    click Demo "https://github.com/neuromorphiccomputing/softhebb/blob/main/demo.py"
    click TrainCLI "https://github.com/neuromorphiccomputing/softhebb/blob/main/train.py"
    click MultiLayerCLI "https://github.com/neuromorphiccomputing/softhebb/blob/main/multi_layer.py"
    click RaySearchCLI "https://github.com/neuromorphiccomputing/softhebb/blob/main/ray_search.py"
    click PostHoc "https://github.com/neuromorphiccomputing/softhebb/blob/main/post_hoc_loss.py"
    click GraphsNotebook "https://github.com/neuromorphiccomputing/softhebb/blob/main/graphs.ipynb"
    click Presets "https://github.com/neuromorphiccomputing/softhebb/blob/main/presets.json"
    click CondaEnv "https://github.com/neuromorphiccomputing/softhebb/blob/main/environment_pytorch==1.7.1.yml"
    click Dataset "https://github.com/neuromorphiccomputing/softhebb/blob/main/dataset.py"
    click Utils "https://github.com/neuromorphiccomputing/softhebb/blob/main/utils.py"
    click LayerBlocks "https://github.com/neuromorphiccomputing/softhebb/blob/main/layer.py"
    click HebbLinear "https://github.com/neuromorphiccomputing/softhebb/blob/main/hebblinear.py"
    click HebbConv "https://github.com/neuromorphiccomputing/softhebb/blob/main/hebbconv.py"
    click ActivationFuncs "https://github.com/neuromorphiccomputing/softhebb/blob/main/activation.py"
    click ModelWrapper "https://github.com/neuromorphiccomputing/softhebb/blob/main/model.py"
    click OneEpoch "https://github.com/neuromorphiccomputing/softhebb/blob/main/engine.py"
    click MultiEpoch "https://github.com/neuromorphiccomputing/softhebb/blob/main/train.py"
    click Logger "https://github.com/neuromorphiccomputing/softhebb/blob/main/log.py"
    click RaySearch "https://github.com/neuromorphiccomputing/softhebb/blob/main/ray_search.py"
    click NBUtils "https://github.com/neuromorphiccomputing/softhebb/blob/main/nb_utils.py"

    %% Styles
    classDef scripts fill:#ffe4b3,stroke:#cc9500;
    classDef config fill:#d0e1f9,stroke:#3b70b2;
    classDef data fill:#b3e5ff,stroke:#3588a0;
    classDef model fill:#c8e6c9,stroke:#4a7c40;
    classDef training fill:#ffe0b2,stroke:#b26a00;
    classDef search fill:#e1bee7,stroke:#7d4a7b;
    classDef vis fill:#d1c4e9,stroke:#5e3b8b;
    classDef datastore fill:#aed581,stroke:#6c8a23;
```
