# humancentricart/memo architecture

> Architecture diagram of the GitHub repository humancentricart/memo, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-05-16). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/humancentricart/memo
- Repository: https://github.com/humancentricart/memo

## Overview

This repository is a research codebase for **MeMo**, a language-modeling approach centered on **associative memory** and a **memorization-first** workflow. It is not organized like a production app; the architecture is dominated by three alternative implementations of the same core idea, plus notebooks for experiments and demonstrations.

1. **Top-level architecture**
   - The repo has one standalone reference implementation, two package-based implementations, and a set of experiment notebooks/results.
   - The three implementations are:
     - `MeMoCMM.py` — a compact PyTorch-only version using direct vector/matrix operations.
     - `MeMoPyTorch/` — a modular PyTorch version built from neural-network components.
     - `MeMoHF/` — a HuggingFace-compatible version shaped like a transformer model.

2. **Core conceptual pipeline**
   - The central flow is: **text input → tokenization → embedding/representation → memorization in associative memory → retrieval / forgetting / evaluation**.
   - The repository is explicitly about separating a **memorization phase** from a later **learning phase**, so “memory” is the key architectural boundary.

3. **Reference implementation: `MeMoCMM.py`**
   - This is the simplest implementation and likely serves as the baseline for understanding the algorithm.
   - It appears to express MeMo with low-level tensor math rather than a layered model abstraction.
   - For diagramming purposes, treat this as the “minimal core” version of the algorithm.

4. **Modular PyTorch package: `MeMoPyTorch/`**
   - This package is the cleanest place to look for a componentized architecture.
   - It contains separate modules for:
     - `modelling_memo.py` — the model assembly/orchestration layer.
     - `modelling_memo_layer.py` — the core memory/model layer.
     - `modelling_memo_embedding.py` — embedding and representation logic.
     - `modelling_memo_tokenizer.py` — text-to-token conversion.
     - `evaluating_memo.py` — evaluation helpers.
     - `modelling_memo_exception.py` — domain-specific error handling.
   - This version is meant to be “ready for learning after memorization,” so it likely exposes a clearer interface for downstream training loops.

5. **HuggingFace-compatible package: `MeMoHF/`**
   - This package mirrors the PyTorch structure but adapts MeMo to HuggingFace-style model conventions.
   - It includes:
     - `modelling_memo.py`
     - `modelling_memo_configuration.py`
     - `modelling_memo_embedding.py`
     - `modelling_memo_layer.py`
     - `modelling_memo_tokenizer.py`
     - `evaluating_memo.py`
     - `modelling_memo_exception.py`
   - The presence of a configuration module suggests this version plugs into model/config ecosystems used by transformer tooling.

6. **Shared subsystems across the packages**
   - The repeated file names across `MeMoPyTorch` and `MeMoHF` indicate a shared conceptual decomposition:
     - tokenizer boundary
     - embedding boundary
     - memory/layer boundary
     - model wrapper
     - evaluation boundary
   - These are the main architectural boxes to show in a diagram, because they define the flow of data and responsibilities.

7. **Memory mechanism as the central domain boundary**
   - The repository’s differentiator is the **associative memory** mechanism, which appears to be the heart of both memorization and retrieval.
   - “Forgetting” is also treated as a first-class operation, which means the model is not just a static encoder but a mutable memory system.
   - In a diagram, this memory component should sit between representation generation and output/evaluation.

8. **Evaluation subsystem**
   - Both package versions include `evaluating_memo.py`, implying that evaluation is implemented alongside the model rather than externally.
   - This suggests the repo uses evaluation to measure memorization/retrieval behavior and probably to compare against experimental settings from the paper.

9. **Notebook-based experimentation layer**
   - The `Experiments/` directory contains notebooks and result spreadsheets, showing that much of the repo is exploratory and empirical.
   - The `PlayingWithMemo*.ipynb` notebooks serve as demonstrations of memorizing, retrieving, and forgetting across the different implementations.
   - `Playground.py` appears to be a script entry point for interactive experimentation outside notebooks.

10. **Research workflow and outputs**
    - The `.xlsx` files in `Experiments/` are output artifacts, likely recording metrics or tables from the paper.
    - The repository therefore has a clear separation between:
      - library code implementing MeMo
      - notebook/script experimentation
      - saved experimental results

11. **Boundary between implementation styles**
    - `MeMoCMM.py`, `MeMoPyTorch`, and `MeMoHF` are parallel implementations rather than layers of one stack.
    - They represent different integration targets:
      - pure tensor prototype
      - PyTorch module-based model
      - HuggingFace ecosystem compatibility
    - An architecture diagram should show them as sibling variants, not as dependencies stacked on one another.

12. **External ecosystem relevance**
    - The only materially important external ecosystems are **PyTorch** and **HuggingFace**, because they shape the package structure and intended reuse.
    - There are no signs of a backend service, database, or API infrastructure; this is primarily a model-research repository.

13. **What to emphasize in a diagram**
    - Put **tokenizer → embedding → memory/layer → model wrapper → evaluation** as the core flow.
    - Surround that with three implementation variants: CMM, PyTorch, HuggingFace.
    - Show notebooks/experiments as consumers of the library rather than as part of the runtime model path.

14. **Overall characterization**
    - This repo is best understood as a **research lab for associative-memory language models**, with multiple implementation tiers and a strong emphasis on experimentation.
    - The most important architectural idea is the separation of **memorization mechanics** from **model integration**, which is reflected consistently across the code organization.

Main entry points: [`MeMoPyTorch/modelling_memo.py`](https://github.com/humancentricart/memo/blob/master/MeMoPyTorch/modelling_memo.py) (Model wrapper), [`MeMoHF/modelling_memo.py`](https://github.com/humancentricart/memo/blob/master/MeMoHF/modelling_memo.py) (HF model), [`MeMoPyTorch/modelling_memo_tokenizer.py`](https://github.com/humancentricart/memo/blob/master/MeMoPyTorch/modelling_memo_tokenizer.py) (Tokenizer), [`Experiments/MemorizationExperiments.ipynb`](https://github.com/humancentricart/memo/blob/master/Experiments/MemorizationExperiments.ipynb) (Notebooks)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_impls["Implementations"]
  node_memo_cmm["MeMo CMM<br/>reference impl<br/>[MeMoCMM.py]"]
  node_memo_pytorch_model["Model wrapper<br/>orchestrator<br/>[modelling_memo.py]"]
  node_memo_pytorch_layer["Memory layer<br/>associative memory"]
  node_memo_pytorch_embed["Embedding<br/>representation"]
  node_memo_pytorch_tokenizer["Tokenizer<br/>text interface"]
  node_memo_pytorch_eval["Evaluation<br/>metrics<br/>[evaluating_memo.py]"]
  node_memo_pytorch_exc["Exceptions<br/>errors"]
  node_memo_hf_model["HF model<br/>transformer model<br/>[modelling_memo.py]"]
  node_memo_hf_config["Config"]
  node_memo_hf_layer["Memory layer<br/>associative memory"]
  node_memo_hf_embed["Embedding<br/>representation"]
  node_memo_hf_tokenizer["Tokenizer<br/>text interface"]
  node_memo_hf_eval["Evaluation<br/>metrics<br/>[evaluating_memo.py]"]
  node_memo_hf_exc["Exceptions<br/>errors"]
end

subgraph group_research["Research"]
  node_experiments_nb["Notebooks<br/>experiments"]
  node_experiments_results[("Results<br/>artifacts")]
  node_playground["Playground<br/>script<br/>[Playground.py]"]
  node_playing_notebooks["Demo notebooks<br/>experiments"]
end

node_memo_cmm -.->|"conceptual flow"| node_memo_pytorch_tokenizer
node_memo_pytorch_tokenizer -->|"tokens"| node_memo_pytorch_embed
node_memo_pytorch_embed -->|"representations"| node_memo_pytorch_layer
node_memo_pytorch_layer -->|"state"| node_memo_pytorch_model
node_memo_pytorch_model -->|"evaluate"| node_memo_pytorch_eval
node_memo_pytorch_tokenizer -->|"input"| node_memo_pytorch_model
node_memo_pytorch_exc -.->|"errors"| node_memo_pytorch_model
node_memo_hf_config -->|"configure"| node_memo_hf_model
node_memo_hf_tokenizer -->|"tokens"| node_memo_hf_embed
node_memo_hf_embed -->|"representations"| node_memo_hf_layer
node_memo_hf_layer -->|"state"| node_memo_hf_model
node_memo_hf_model -->|"evaluate"| node_memo_hf_eval
node_memo_hf_exc -.->|"errors"| node_memo_hf_model
node_memo_hf_model -.->|"parallel variant"| node_memo_pytorch_model
node_experiments_nb -->|"uses"| node_memo_pytorch_model
node_experiments_nb -->|"uses"| node_memo_hf_model
node_playing_notebooks -->|"demonstrates"| node_memo_pytorch_model
node_playing_notebooks -->|"demonstrates"| node_memo_cmm
node_playground -->|"drives"| node_memo_pytorch_model
node_experiments_results -.->|"outputs"| node_experiments_nb

click node_memo_cmm "https://github.com/humancentricart/memo/blob/master/MeMoCMM.py"
click node_memo_pytorch_model "https://github.com/humancentricart/memo/blob/master/MeMoPyTorch/modelling_memo.py"
click node_memo_pytorch_layer "https://github.com/humancentricart/memo/blob/master/MeMoPyTorch/modelling_memo_layer.py"
click node_memo_pytorch_embed "https://github.com/humancentricart/memo/blob/master/MeMoPyTorch/modelling_memo_embedding.py"
click node_memo_pytorch_tokenizer "https://github.com/humancentricart/memo/blob/master/MeMoPyTorch/modelling_memo_tokenizer.py"
click node_memo_pytorch_eval "https://github.com/humancentricart/memo/blob/master/MeMoPyTorch/evaluating_memo.py"
click node_memo_pytorch_exc "https://github.com/humancentricart/memo/blob/master/MeMoPyTorch/modelling_memo_exception.py"
click node_memo_hf_model "https://github.com/humancentricart/memo/blob/master/MeMoHF/modelling_memo.py"
click node_memo_hf_config "https://github.com/humancentricart/memo/blob/master/MeMoHF/modelling_memo_configuration.py"
click node_memo_hf_layer "https://github.com/humancentricart/memo/blob/master/MeMoHF/modelling_memo_layer.py"
click node_memo_hf_embed "https://github.com/humancentricart/memo/blob/master/MeMoHF/modelling_memo_embedding.py"
click node_memo_hf_tokenizer "https://github.com/humancentricart/memo/blob/master/MeMoHF/modelling_memo_tokenizer.py"
click node_memo_hf_eval "https://github.com/humancentricart/memo/blob/master/MeMoHF/evaluating_memo.py"
click node_memo_hf_exc "https://github.com/humancentricart/memo/blob/master/MeMoHF/modelling_memo_exception.py"
click node_experiments_nb "https://github.com/humancentricart/memo/blob/master/Experiments/MemorizationExperiments.ipynb"
click node_experiments_results "https://github.com/humancentricart/memo/blob/master/Experiments/MemorizationResults.xlsx"
click node_playground "https://github.com/humancentricart/memo/blob/master/Playground.py"
click node_playing_notebooks "https://github.com/humancentricart/memo/blob/master/PlayingWithMemo.ipynb"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_memo_cmm,node_memo_pytorch_model,node_memo_pytorch_layer,node_memo_pytorch_embed,node_memo_pytorch_tokenizer,node_memo_pytorch_eval,node_memo_pytorch_exc,node_memo_hf_model,node_memo_hf_config,node_memo_hf_layer,node_memo_hf_embed,node_memo_hf_tokenizer,node_memo_hf_eval,node_memo_hf_exc toneBlue
class node_experiments_nb,node_experiments_results,node_playground,node_playing_notebooks toneAmber
```

## Components

### Implementations

Parallel MeMo implementations for different integration targets.

- [MeMo CMM](https://github.com/humancentricart/memo/blob/master/MeMoCMM.py): reference impl · Compact PyTorch-only baseline using direct tensor operations. · `MeMoCMM.py`
- [Model wrapper](https://github.com/humancentricart/memo/blob/master/MeMoPyTorch/modelling_memo.py): orchestrator · Assembles the modular PyTorch MeMo pipeline. · `MeMoPyTorch/modelling_memo.py`
- [Memory layer](https://github.com/humancentricart/memo/blob/master/MeMoPyTorch/modelling_memo_layer.py): associative memory · Core memorization, retrieval, and forgetting logic for the PyTorch package. · `MeMoPyTorch/modelling_memo_layer.py`
- [Embedding](https://github.com/humancentricart/memo/blob/master/MeMoPyTorch/modelling_memo_embedding.py): representation · Converts tokens into internal representations for memorization. · `MeMoPyTorch/modelling_memo_embedding.py`
- [Tokenizer](https://github.com/humancentricart/memo/blob/master/MeMoPyTorch/modelling_memo_tokenizer.py): text interface · Text-to-token boundary for the modular PyTorch implementation. · `MeMoPyTorch/modelling_memo_tokenizer.py`
- [Evaluation](https://github.com/humancentricart/memo/blob/master/MeMoPyTorch/evaluating_memo.py): metrics · Evaluation helpers for memorization and retrieval behavior. · `MeMoPyTorch/evaluating_memo.py`
- [Exceptions](https://github.com/humancentricart/memo/blob/master/MeMoPyTorch/modelling_memo_exception.py): errors · Domain-specific errors for the PyTorch implementation. · `MeMoPyTorch/modelling_memo_exception.py`
- [HF model](https://github.com/humancentricart/memo/blob/master/MeMoHF/modelling_memo.py): transformer model · HuggingFace-style MeMo wrapper aligned with model conventions. · `MeMoHF/modelling_memo.py`
- [Config](https://github.com/humancentricart/memo/blob/master/MeMoHF/modelling_memo_configuration.py): configuration · Model configuration for HuggingFace integration. · `MeMoHF/modelling_memo_configuration.py`
- [Memory layer](https://github.com/humancentricart/memo/blob/master/MeMoHF/modelling_memo_layer.py): associative memory · Core MeMo memory layer adapted to HuggingFace packaging. · `MeMoHF/modelling_memo_layer.py`
- [Embedding](https://github.com/humancentricart/memo/blob/master/MeMoHF/modelling_memo_embedding.py): representation · Representation logic for the HuggingFace-compatible variant. · `MeMoHF/modelling_memo_embedding.py`
- [Tokenizer](https://github.com/humancentricart/memo/blob/master/MeMoHF/modelling_memo_tokenizer.py): text interface · Text-to-token boundary for the HuggingFace-compatible variant. · `MeMoHF/modelling_memo_tokenizer.py`
- [Evaluation](https://github.com/humancentricart/memo/blob/master/MeMoHF/evaluating_memo.py): metrics · Evaluation helpers for the HuggingFace-compatible variant. · `MeMoHF/evaluating_memo.py`
- [Exceptions](https://github.com/humancentricart/memo/blob/master/MeMoHF/modelling_memo_exception.py): errors · Domain-specific errors for the HuggingFace-compatible variant. · `MeMoHF/modelling_memo_exception.py`

### Research

Notebooks, scripts, and saved experiment outputs.

- [Notebooks](https://github.com/humancentricart/memo/blob/master/Experiments/MemorizationExperiments.ipynb): experiments · Interactive memorization and retrieval demonstrations. · `Experiments/MemorizationExperiments.ipynb`
- [Results](https://github.com/humancentricart/memo/blob/master/Experiments/MemorizationResults.xlsx): artifacts · Saved experiment tables and metrics. · `Experiments/MemorizationResults.xlsx`
- [Playground](https://github.com/humancentricart/memo/blob/master/Playground.py): script · Interactive script for trying MeMo workflows outside notebooks. · `Playground.py`
- [Demo notebooks](https://github.com/humancentricart/memo/blob/master/PlayingWithMemo.ipynb): experiments · Notebook demonstrations across the PyTorch and HF variants. · `PlayingWithMemo.ipynb`

## Connections

- MeMo CMM → Tokenizer: conceptual flow · The baseline expresses the same text-to-memory pipeline in minimal tensor form.
- Tokenizer → Embedding: tokens · Tokenized text is converted into internal representations.
- Embedding → Memory layer: representations · Representations are stored and manipulated by associative memory.
- Memory layer → Model wrapper: state · The model wrapper coordinates memory behavior and learning workflow.
- Model wrapper → Evaluation: evaluate · Model outputs and memory behavior are measured through evaluation helpers.
- Tokenizer → Model wrapper: input · Tokenizer feeds the assembled PyTorch model.
- Exceptions → Model wrapper: errors · Domain exceptions support the PyTorch model boundary.
- Config → HF model: configure · Configuration drives HuggingFace model construction.
- Tokenizer → Embedding: tokens · Tokenized inputs become representations.
- Embedding → Memory layer: representations · Representations are routed into associative memory.
- Memory layer → HF model: state · The HF model wrapper coordinates memory and transformer-style integration.
- HF model → Evaluation: evaluate · Evaluation measures memorization, retrieval, and forgetting behavior.
- Exceptions → HF model: errors · Domain exceptions support the HF model boundary.
- HF model → Model wrapper: parallel variant · The HF package mirrors the modular PyTorch architecture with ecosystem-specific packaging.
- Notebooks → Model wrapper: uses · Notebooks consume the modular implementation for experiments.
- Notebooks → HF model: uses · Notebooks also compare the HuggingFace-compatible variant.
- Demo notebooks → Model wrapper: demonstrates · Demo notebooks exercise memorization and forgetting workflows.
- Demo notebooks → MeMo CMM: demonstrates · Some notebooks explore the reference implementation.
- Playground → Model wrapper: drives · The playground script provides an interactive entry point.
- Results → Notebooks: outputs · Spreadsheets record the outcomes of notebook-driven experiments.
