# bartbussmann/matryoshka_sae architecture

> Architecture diagram of the GitHub repository bartbussmann/matryoshka_sae, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-06-04). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/bartbussmann/matryoshka_sae
- Repository: https://github.com/bartbussmann/matryoshka_sae

## Overview

##### Project type and purpose
- A Python-based ML training pipeline for “Matryoshka” sparse autoencoders.
- Core goal: define, train, log and persist SAE models with multi-level latent regularization.

##### Main components
- Entry point (main.py)
- Configuration loader (config.py)
- Model definition (sae.py)
- Training engine (training.py)
- Activation store (activation_store.py)
- Utilities (utils.py)
- Logging/monitoring (logs.py)
- External dependencies (transformer_lens, filesystem)

##### Interactions & data flow
[main.py] → loads → [config.py] → returns hyperparams
[main.py] → instantiates → [SAE model (sae.py)]
[main.py] → calls → [Training engine (training.py)]
[Training engine] ↔ uses ↔ [Model], [Utils]
[Training engine] → writes → [Activation store] & [Logs]
[Activation store] → persists → latent activations to disk
[Logs module] → records → training progress, metrics
All modules import common helpers from [utils.py]

##### Architectural patterns & principles
- Modular separation: config, modeling, training, persistence, logging
- Single-process, script-driven pipeline
- Clear data-flow: config → model → train → store/log

##### Elements
- Arrowheads for direction
- Optional grouping boundary (“ML Core”) around model+training components

This compact layout will capture all essential components, their responsibilities, and the data/control flow in the Matryoshka SAE codebase.

## Diagram (Mermaid)

```mermaid
flowchart TB
    %% Entry Point
    main["main.py"]:::entry
    %% Configuration & Utilities
    config["config.py"]:::infra
    utils["utils.py"]:::util
    %% ML Core Components
    subgraph "ML Core"
        sae["sae.py"]:::core
        training["training.py"]:::core
    end
    %% Infrastructure
    logs["logs.py"]:::infra
    store["activation_store.py"]:::infra
    %% External Library
    extTransformer["transformer_lens"]:::external
    %% Outputs
    outputModel["Trained SAE Model"]:::output
    outputAct["Activation Logs"]:::output

    %% Data & Control Flow
    main -->|"load config"| config
    main -->|"instantiate model"| sae
    main -->|"start training"| training
    training -->|"uses model"| sae
    training -->|"uses utils"| utils
    sae -->|"imports"| extTransformer
    training -->|"save activations"| store
    training -->|"log metrics"| logs
    training ---|"output model"| outputModel
    store ---|"persist activations"| outputAct
    logs ---|"persist logs"| outputAct

    %% Click Events
    click main "https://github.com/bartbussmann/matryoshka_sae/blob/main/main.py"
    click config "https://github.com/bartbussmann/matryoshka_sae/blob/main/config.py"
    click sae "https://github.com/bartbussmann/matryoshka_sae/blob/main/sae.py"
    click training "https://github.com/bartbussmann/matryoshka_sae/blob/main/training.py"
    click store "https://github.com/bartbussmann/matryoshka_sae/blob/main/activation_store.py"
    click logs "https://github.com/bartbussmann/matryoshka_sae/blob/main/logs.py"
    click utils "https://github.com/bartbussmann/matryoshka_sae/blob/main/utils.py"

    %% Styling
    classDef entry fill:#FFA500,stroke:#333,stroke-width:1px
    classDef core fill:#ADD8E6,stroke:#333,stroke-width:1px
    classDef infra fill:#90EE90,stroke:#333,stroke-width:1px
    classDef util fill:#D3D3D3,stroke:#333,stroke-width:1px
    classDef external fill:#DDA0DD,stroke:#333,stroke-width:1px
    classDef output fill:#FFFFE0,stroke:#333,stroke-width:1px

    %% Legend
    subgraph "Legend"
        legendEntry1["main.py (Entry Point)"]:::entry
        legendEntry2["sae.py, training.py (Core ML)"]:::core
        legendEntry3["config.py, logs.py, activation_store.py (Infra)"]:::infra
        legendEntry4["utils.py (Utilities)"]:::util
        legendEntry5["transformer_lens (External)"]:::external
        legendEntry6["Outputs"]:::output
    end
```
