How alignment-research-center/mlp_cumulant_propagation works
This is a research implementation of cumulant propagation for wide random MLPs. The main workflow configures an MLP and input cumulants, propagates cumulants layer by layer, then reads the estimated output (optionally as tensors in harmonic form). The repository also supports factorized propagation, symbolic propagation, and width-sweep experiments with summary and plotting. Relationships among modules are based on README statements and SOURCE INDEX references; unsampled implementation details are not inferred.
Main entry points: kprop_harmonic.py Harmonic kprop · kprop_by_width.py Width experiments · kprop.py Symbolic kprop · plot.py Experiment plots
Components 23
MLP and inputs
- MLP model
src/mlp_kprop/mlp.py - Input cumulants
Propagation runtime
- Harmonic kprop
src/mlp_kprop/kprop_harmonic.py - Harmonic tensors
src/mlp_kprop/harmonic.py - Diagonal slices
src/mlp_kprop/diagslice.py - Order-3 factoring
src/mlp_kprop/factor_k3.py - Order-4 factoring
src/mlp_kprop/factor_k4.py - Activation coefficients
src/mlp_kprop/wick.py - Partition combinatorics
src/mlp_kprop/partitions.py - Tensor operations
src/mlp_kprop/tensor_utils.py - Sample cumulants
src/mlp_kprop/cumulants.py - Legacy DS kprop
src/mlp_kprop/kprop_ds.py
Symbolic propagation
- Symbolic kprop
src/mlp_kprop/symb/kprop.py - Symbolic slices
src/mlp_kprop/symb/diagslice.py - Lazy tensors
src/mlp_kprop/symb/lazy_tensor.py - Tensor networks
src/mlp_kprop/symb/network.py - Network pruning
src/mlp_kprop/symb/pruning.py
Experiments and analysis
- Width experiments
scripts/kprop_by_width.py - Result summaries
scripts/form_df.py - Experiment plots
scripts/plot.py - Plot utilities
src/mlp_kprop/plotting_utils.py - FLOP accounting
src/mlp_kprop/flop_utils.py
Other components
- Researcher
Connections 31
- Researcher → MLP model: configures
README.md - Researcher → Input cumulants: supplies
README.md - MLP model → Harmonic kprop: model input
README.md - Input cumulants → Harmonic kprop: initializes
README.md - Harmonic kprop → Harmonic tensors: uses representation
src/mlp_kprop/kprop_harmonic.py - Harmonic kprop → Diagonal slices: uses representation
src/mlp_kprop/kprop_harmonic.py - Harmonic kprop → Activation coefficients: uses coefficients
src/mlp_kprop/kprop_harmonic.py - Harmonic kprop → Partition combinatorics: uses partitions
src/mlp_kprop/kprop_harmonic.py - Harmonic kprop → Tensor operations: uses operations
src/mlp_kprop/kprop_harmonic.py - Harmonic kprop → Order-3 factoring: delegates factoring
src/mlp_kprop/kprop_harmonic.py - Harmonic kprop → Order-4 factoring: delegates factoring
src/mlp_kprop/kprop_harmonic.py - Sample cumulants → Diagonal slices: computes in
src/mlp_kprop/cumulants.py - Sample cumulants → Partition combinatorics: uses partitions
src/mlp_kprop/cumulants.py - Legacy DS kprop → Diagonal slices: uses representation
src/mlp_kprop/kprop_ds.py - Legacy DS kprop → Activation coefficients: uses coefficients
src/mlp_kprop/kprop_ds.py - Symbolic kprop → Symbolic slices: uses symbolic slices
src/mlp_kprop/symb/kprop.py - Symbolic kprop → Lazy tensors: uses lazy tensors
src/mlp_kprop/symb/kprop.py - Symbolic kprop → Tensor networks: uses networks
src/mlp_kprop/symb/kprop.py - Symbolic slices → Tensor networks: uses networks
src/mlp_kprop/symb/diagslice.py - Symbolic slices → Lazy tensors: uses lazy tensors
src/mlp_kprop/symb/diagslice.py - Network pruning → Tensor networks: prunes networks
src/mlp_kprop/symb/pruning.py - Network pruning → Lazy tensors: uses lazy tensors
src/mlp_kprop/symb/pruning.py - Width experiments → MLP model: constructs models
scripts/kprop_by_width.py - Width experiments → Harmonic kprop: runs propagation
scripts/kprop_by_width.py - Width experiments → Sample cumulants: computes samples
scripts/kprop_by_width.py - Width experiments → Legacy DS kprop: runs variant
scripts/kprop_by_width.py - Result summaries → Width experiments: consumes results
scripts/form_df.py - Result summaries → FLOP accounting: uses FLOP counts
scripts/form_df.py - Experiment plots → Result summaries: plots summaries
README.md - Experiment plots → Plot utilities: uses utilities
scripts/plot.py - Experiment plots → FLOP accounting: uses FLOP counts
scripts/plot.py