How bartbussmann/matryoshka_sae works
Project type and purpose
• A Python-based ML training pipeline for “Matryoshka” sparse autoencoders.
• Core goal: define, train, log and persist SAE models with multi-level latent regularization.
Main components
• Entry point (main.py)
• Configuration loader (config.py)
• Model definition (sae.py)
• Training engine (training.py)
• Activation store (activation_store.py)
• Utilities (utils.py)
• Logging/monitoring (logs.py)
• External dependencies (transformer_lens, filesystem)
Interactions & data flow
[main.py] → loads → [config.py] → returns hyperparams
[main.py] → instantiates → [SAE model (sae.py)]
[main.py] → calls → [Training engine (training.py)]
[Training engine] ↔ uses ↔ [Model], [Utils]
[Training engine] → writes → [Activation store] & [Logs]
[Activation store] → persists → latent activations to disk
[Logs module] → records → training progress, metrics
All modules import common helpers from [utils.py]
Architectural patterns & principles
• Modular separation: config, modeling, training, persistence, logging
• Single-process, script-driven pipeline
• Clear data-flow: config → model → train → store/log
Elements
• Arrowheads for direction
• Optional grouping boundary (“ML Core”) around model+training components
This compact layout will capture all essential components, their responsibilities, and the data/control flow in the Matryoshka SAE codebase.