# alignment-research-center/mlp_cumulant_propagation architecture

> Architecture diagram of the GitHub repository alignment-research-center/mlp_cumulant_propagation, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-10-07). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/alignment-research-center/mlp_cumulant_propagation
- Repository: https://github.com/alignment-research-center/mlp_cumulant_propagation

## Overview

This is a research implementation of cumulant propagation for wide random MLPs. The main workflow configures an MLP and input cumulants, propagates cumulants layer by layer, then reads the estimated output (optionally as tensors in harmonic form). The repository also supports factorized propagation, symbolic propagation, and width-sweep experiments with summary and plotting. Relationships among modules are based on README statements and SOURCE INDEX references; unsampled implementation details are not inferred.

Main entry points: [`src/mlp_kprop/kprop_harmonic.py`](https://github.com/alignment-research-center/mlp_cumulant_propagation/blob/main/src/mlp_kprop/kprop_harmonic.py) (Harmonic kprop), [`scripts/kprop_by_width.py`](https://github.com/alignment-research-center/mlp_cumulant_propagation/blob/main/scripts/kprop_by_width.py) (Width experiments), [`src/mlp_kprop/symb/kprop.py`](https://github.com/alignment-research-center/mlp_cumulant_propagation/blob/main/src/mlp_kprop/symb/kprop.py) (Symbolic kprop), [`scripts/plot.py`](https://github.com/alignment-research-center/mlp_cumulant_propagation/blob/main/scripts/plot.py) (Experiment plots)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_model["MLP and inputs"]
  node_mlp["MLP model<br/>[mlp.py]"]
  node_input_cumulants["Input cumulants"]
end

subgraph group_runtime["Propagation runtime"]
  node_kprop["Harmonic kprop<br/>[kprop_harmonic.py]"]
  node_harmonic["Harmonic tensors<br/>[harmonic.py]"]
  node_diagslice["Diagonal slices<br/>[diagslice.py]"]
  node_factor_k3["Order-3 factoring<br/>[factor_k3.py]"]
  node_factor_k4["Order-4 factoring<br/>[factor_k4.py]"]
  node_wick["Activation coefficients<br/>[wick.py]"]
  node_partitions["Partition combinatorics<br/>[partitions.py]"]
  node_tensor_ops["Tensor operations<br/>[tensor_utils.py]"]
  node_cumulants["Sample cumulants<br/>[cumulants.py]"]
  node_legacy_ds["Legacy DS kprop<br/>[kprop_ds.py]"]
end

subgraph group_symbolic["Symbolic propagation"]
  node_symbolic_kprop["Symbolic kprop<br/>[kprop.py]"]
  node_symbolic_ds["Symbolic slices<br/>[diagslice.py]"]
  node_lazy_tensor["Lazy tensors<br/>[lazy_tensor.py]"]
  node_network["Tensor networks<br/>[network.py]"]
  node_pruning["Network pruning<br/>[pruning.py]"]
end

subgraph group_experiments["Experiments and analysis"]
  node_runner["Width experiments<br/>[kprop_by_width.py]"]
  node_dataframe["Result summaries<br/>[form_df.py]"]
  node_plotting["Experiment plots<br/>[plot.py]"]
  node_plot_utils["Plot utilities<br/>[plotting_utils.py]"]
  node_flops["FLOP accounting<br/>[flop_utils.py]"]
end

node_researcher(("Researcher"))

node_researcher -->|"configures"| node_mlp
node_researcher -->|"supplies"| node_input_cumulants
node_mlp -->|"model input"| node_kprop
node_input_cumulants -->|"initializes"| node_kprop
node_kprop -->|"uses representation"| node_harmonic
node_kprop -->|"uses representation"| node_diagslice
node_kprop -->|"uses coefficients"| node_wick
node_kprop -->|"uses partitions"| node_partitions
node_kprop -->|"uses operations"| node_tensor_ops
node_kprop -->|"delegates factoring"| node_factor_k3
node_kprop -->|"delegates factoring"| node_factor_k4
node_cumulants -->|"computes in"| node_diagslice
node_cumulants -->|"uses partitions"| node_partitions
node_legacy_ds -->|"uses representation"| node_diagslice
node_legacy_ds -->|"uses coefficients"| node_wick
node_symbolic_kprop -->|"uses symbolic slices"| node_symbolic_ds
node_symbolic_kprop -->|"uses lazy tensors"| node_lazy_tensor
node_symbolic_kprop -->|"uses networks"| node_network
node_symbolic_ds -->|"uses networks"| node_network
node_symbolic_ds -->|"uses lazy tensors"| node_lazy_tensor
node_pruning -->|"prunes networks"| node_network
node_pruning -->|"uses lazy tensors"| node_lazy_tensor
node_runner -->|"constructs models"| node_mlp
node_runner -->|"runs propagation"| node_kprop
node_runner -->|"computes samples"| node_cumulants
node_runner -->|"runs variant"| node_legacy_ds
node_dataframe -->|"consumes results"| node_runner
node_dataframe -->|"uses FLOP counts"| node_flops
node_plotting -->|"plots summaries"| node_dataframe
node_plotting -->|"uses utilities"| node_plot_utils
node_plotting -->|"uses FLOP counts"| node_flops

click node_mlp "https://github.com/alignment-research-center/mlp_cumulant_propagation/blob/main/src/mlp_kprop/mlp.py"
click node_kprop "https://github.com/alignment-research-center/mlp_cumulant_propagation/blob/main/src/mlp_kprop/kprop_harmonic.py"
click node_harmonic "https://github.com/alignment-research-center/mlp_cumulant_propagation/blob/main/src/mlp_kprop/harmonic.py"
click node_diagslice "https://github.com/alignment-research-center/mlp_cumulant_propagation/blob/main/src/mlp_kprop/diagslice.py"
click node_factor_k3 "https://github.com/alignment-research-center/mlp_cumulant_propagation/blob/main/src/mlp_kprop/factor_k3.py"
click node_factor_k4 "https://github.com/alignment-research-center/mlp_cumulant_propagation/blob/main/src/mlp_kprop/factor_k4.py"
click node_wick "https://github.com/alignment-research-center/mlp_cumulant_propagation/blob/main/src/mlp_kprop/wick.py"
click node_partitions "https://github.com/alignment-research-center/mlp_cumulant_propagation/blob/main/src/mlp_kprop/partitions.py"
click node_tensor_ops "https://github.com/alignment-research-center/mlp_cumulant_propagation/blob/main/src/mlp_kprop/tensor_utils.py"
click node_cumulants "https://github.com/alignment-research-center/mlp_cumulant_propagation/blob/main/src/mlp_kprop/cumulants.py"
click node_legacy_ds "https://github.com/alignment-research-center/mlp_cumulant_propagation/blob/main/src/mlp_kprop/kprop_ds.py"
click node_symbolic_kprop "https://github.com/alignment-research-center/mlp_cumulant_propagation/blob/main/src/mlp_kprop/symb/kprop.py"
click node_symbolic_ds "https://github.com/alignment-research-center/mlp_cumulant_propagation/blob/main/src/mlp_kprop/symb/diagslice.py"
click node_lazy_tensor "https://github.com/alignment-research-center/mlp_cumulant_propagation/blob/main/src/mlp_kprop/symb/lazy_tensor.py"
click node_network "https://github.com/alignment-research-center/mlp_cumulant_propagation/blob/main/src/mlp_kprop/symb/network.py"
click node_pruning "https://github.com/alignment-research-center/mlp_cumulant_propagation/blob/main/src/mlp_kprop/symb/pruning.py"
click node_runner "https://github.com/alignment-research-center/mlp_cumulant_propagation/blob/main/scripts/kprop_by_width.py"
click node_dataframe "https://github.com/alignment-research-center/mlp_cumulant_propagation/blob/main/scripts/form_df.py"
click node_plotting "https://github.com/alignment-research-center/mlp_cumulant_propagation/blob/main/scripts/plot.py"
click node_plot_utils "https://github.com/alignment-research-center/mlp_cumulant_propagation/blob/main/src/mlp_kprop/plotting_utils.py"
click node_flops "https://github.com/alignment-research-center/mlp_cumulant_propagation/blob/main/src/mlp_kprop/flop_utils.py"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_mlp,node_input_cumulants toneBlue
class node_kprop,node_harmonic,node_diagslice,node_factor_k3,node_factor_k4,node_wick,node_partitions,node_tensor_ops,node_cumulants,node_legacy_ds toneAmber
class node_symbolic_kprop,node_symbolic_ds,node_lazy_tensor,node_network,node_pruning toneMint
class node_runner,node_dataframe,node_plotting,node_plot_utils,node_flops toneRose
class node_researcher toneIndigo
```

## Components

### MLP and inputs

- [MLP model](https://github.com/alignment-research-center/mlp_cumulant_propagation/blob/main/src/mlp_kprop/mlp.py): `src/mlp_kprop/mlp.py`
- **Input cumulants**

### Propagation runtime

- [Harmonic kprop](https://github.com/alignment-research-center/mlp_cumulant_propagation/blob/main/src/mlp_kprop/kprop_harmonic.py): `src/mlp_kprop/kprop_harmonic.py`
- [Harmonic tensors](https://github.com/alignment-research-center/mlp_cumulant_propagation/blob/main/src/mlp_kprop/harmonic.py): `src/mlp_kprop/harmonic.py`
- [Diagonal slices](https://github.com/alignment-research-center/mlp_cumulant_propagation/blob/main/src/mlp_kprop/diagslice.py): `src/mlp_kprop/diagslice.py`
- [Order-3 factoring](https://github.com/alignment-research-center/mlp_cumulant_propagation/blob/main/src/mlp_kprop/factor_k3.py): `src/mlp_kprop/factor_k3.py`
- [Order-4 factoring](https://github.com/alignment-research-center/mlp_cumulant_propagation/blob/main/src/mlp_kprop/factor_k4.py): `src/mlp_kprop/factor_k4.py`
- [Activation coefficients](https://github.com/alignment-research-center/mlp_cumulant_propagation/blob/main/src/mlp_kprop/wick.py): `src/mlp_kprop/wick.py`
- [Partition combinatorics](https://github.com/alignment-research-center/mlp_cumulant_propagation/blob/main/src/mlp_kprop/partitions.py): `src/mlp_kprop/partitions.py`
- [Tensor operations](https://github.com/alignment-research-center/mlp_cumulant_propagation/blob/main/src/mlp_kprop/tensor_utils.py): `src/mlp_kprop/tensor_utils.py`
- [Sample cumulants](https://github.com/alignment-research-center/mlp_cumulant_propagation/blob/main/src/mlp_kprop/cumulants.py): `src/mlp_kprop/cumulants.py`
- [Legacy DS kprop](https://github.com/alignment-research-center/mlp_cumulant_propagation/blob/main/src/mlp_kprop/kprop_ds.py): `src/mlp_kprop/kprop_ds.py`

### Symbolic propagation

- [Symbolic kprop](https://github.com/alignment-research-center/mlp_cumulant_propagation/blob/main/src/mlp_kprop/symb/kprop.py): `src/mlp_kprop/symb/kprop.py`
- [Symbolic slices](https://github.com/alignment-research-center/mlp_cumulant_propagation/blob/main/src/mlp_kprop/symb/diagslice.py): `src/mlp_kprop/symb/diagslice.py`
- [Lazy tensors](https://github.com/alignment-research-center/mlp_cumulant_propagation/blob/main/src/mlp_kprop/symb/lazy_tensor.py): `src/mlp_kprop/symb/lazy_tensor.py`
- [Tensor networks](https://github.com/alignment-research-center/mlp_cumulant_propagation/blob/main/src/mlp_kprop/symb/network.py): `src/mlp_kprop/symb/network.py`
- [Network pruning](https://github.com/alignment-research-center/mlp_cumulant_propagation/blob/main/src/mlp_kprop/symb/pruning.py): `src/mlp_kprop/symb/pruning.py`

### Experiments and analysis

- [Width experiments](https://github.com/alignment-research-center/mlp_cumulant_propagation/blob/main/scripts/kprop_by_width.py): `scripts/kprop_by_width.py`
- [Result summaries](https://github.com/alignment-research-center/mlp_cumulant_propagation/blob/main/scripts/form_df.py): `scripts/form_df.py`
- [Experiment plots](https://github.com/alignment-research-center/mlp_cumulant_propagation/blob/main/scripts/plot.py): `scripts/plot.py`
- [Plot utilities](https://github.com/alignment-research-center/mlp_cumulant_propagation/blob/main/src/mlp_kprop/plotting_utils.py): `src/mlp_kprop/plotting_utils.py`
- [FLOP accounting](https://github.com/alignment-research-center/mlp_cumulant_propagation/blob/main/src/mlp_kprop/flop_utils.py): `src/mlp_kprop/flop_utils.py`

### Other components

- **Researcher**

## Connections

- Researcher → MLP model: configures (evidence: `README.md`)
- Researcher → Input cumulants: supplies (evidence: `README.md`)
- MLP model → Harmonic kprop: model input (evidence: `README.md`)
- Input cumulants → Harmonic kprop: initializes (evidence: `README.md`)
- Harmonic kprop → Harmonic tensors: uses representation (evidence: `src/mlp_kprop/kprop_harmonic.py`)
- Harmonic kprop → Diagonal slices: uses representation (evidence: `src/mlp_kprop/kprop_harmonic.py`)
- Harmonic kprop → Activation coefficients: uses coefficients (evidence: `src/mlp_kprop/kprop_harmonic.py`)
- Harmonic kprop → Partition combinatorics: uses partitions (evidence: `src/mlp_kprop/kprop_harmonic.py`)
- Harmonic kprop → Tensor operations: uses operations (evidence: `src/mlp_kprop/kprop_harmonic.py`)
- Harmonic kprop → Order-3 factoring: delegates factoring (evidence: `src/mlp_kprop/kprop_harmonic.py`)
- Harmonic kprop → Order-4 factoring: delegates factoring (evidence: `src/mlp_kprop/kprop_harmonic.py`)
- Sample cumulants → Diagonal slices: computes in (evidence: `src/mlp_kprop/cumulants.py`)
- Sample cumulants → Partition combinatorics: uses partitions (evidence: `src/mlp_kprop/cumulants.py`)
- Legacy DS kprop → Diagonal slices: uses representation (evidence: `src/mlp_kprop/kprop_ds.py`)
- Legacy DS kprop → Activation coefficients: uses coefficients (evidence: `src/mlp_kprop/kprop_ds.py`)
- Symbolic kprop → Symbolic slices: uses symbolic slices (evidence: `src/mlp_kprop/symb/kprop.py`)
- Symbolic kprop → Lazy tensors: uses lazy tensors (evidence: `src/mlp_kprop/symb/kprop.py`)
- Symbolic kprop → Tensor networks: uses networks (evidence: `src/mlp_kprop/symb/kprop.py`)
- Symbolic slices → Tensor networks: uses networks (evidence: `src/mlp_kprop/symb/diagslice.py`)
- Symbolic slices → Lazy tensors: uses lazy tensors (evidence: `src/mlp_kprop/symb/diagslice.py`)
- Network pruning → Tensor networks: prunes networks (evidence: `src/mlp_kprop/symb/pruning.py`)
- Network pruning → Lazy tensors: uses lazy tensors (evidence: `src/mlp_kprop/symb/pruning.py`)
- Width experiments → MLP model: constructs models (evidence: `scripts/kprop_by_width.py`)
- Width experiments → Harmonic kprop: runs propagation (evidence: `scripts/kprop_by_width.py`)
- Width experiments → Sample cumulants: computes samples (evidence: `scripts/kprop_by_width.py`)
- Width experiments → Legacy DS kprop: runs variant (evidence: `scripts/kprop_by_width.py`)
- Result summaries → Width experiments: consumes results (evidence: `scripts/form_df.py`)
- Result summaries → FLOP accounting: uses FLOP counts (evidence: `scripts/form_df.py`)
- Experiment plots → Result summaries: plots summaries (evidence: `README.md`)
- Experiment plots → Plot utilities: uses utilities (evidence: `scripts/plot.py`)
- Experiment plots → FLOP accounting: uses FLOP counts (evidence: `scripts/plot.py`)
