# jbloomaus/saelens architecture

> Architecture diagram of the GitHub repository jbloomaus/saelens, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-10-05). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/jbloomaus/saelens
- Repository: https://github.com/jbloomaus/saelens

## Overview

SAE Lens is a research library for training and analyzing sparse autoencoders on language-model activations, and for loading pretrained SAEs. The main workflow configures a run, obtains model activations, trains an SAE, and evaluates or saves it. Separate capabilities support inference-time model integration and synthetic-data research, including hierarchical synthetic features. The graph reflects README-documented capabilities and SOURCE INDEX references; unsampled implementation details are not inferred as call relationships.

Main entry points: [`sae_lens/llm_sae_training_runner.py`](https://github.com/jbloomaus/saelens/blob/main/sae_lens/llm_sae_training_runner.py) (Language-model training runner), [`sae_lens/evals.py`](https://github.com/jbloomaus/saelens/blob/main/sae_lens/evals.py) (SAE evaluation), [`sae_lens/training/activations_store.py`](https://github.com/jbloomaus/saelens/blob/main/sae_lens/training/activations_store.py) (Activation store), [`sae_lens/saes/sae.py`](https://github.com/jbloomaus/saelens/blob/main/sae_lens/saes/sae.py) (SAE architectures)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_training["SAE training"]
  node_config["Run configuration<br/>[config.py]"]
  node_model_load["Language model loading<br/>[load_model.py]"]
  node_pretokenize["Dataset pretokenization"]
  node_token_batch["Tokenization and batching"]
  node_cache_runner["Activation caching"]
  node_activation_store["Activation store"]
  node_scaler["Activation scaling"]
  node_train_runner["Language-model training runner"]
  node_multi_runner["Multi-SAE training runner"]
  node_trainer["SAE trainer<br/>[sae_trainer.py]"]
  node_multi_trainer["Multi-SAE trainer"]
end

subgraph group_models["SAE models and loading"]
  node_sae["SAE architectures<br/>[sae.py]"]
  node_registry[("Architecture registry<br/>[registry.py]")]
  node_pretrained_load["Pretrained SAE loading"]
  node_pretrained_catalog["Pretrained SAE catalog"]
end

subgraph group_analysis["Inference and analysis"]
  node_evals["SAE evaluation<br/>[evals.py]"]
  node_hooked["Hooked SAE transformer"]
  node_bridge["Transformer bridge"]
  node_neuronpedia["Neuronpedia integration"]
end

subgraph group_synthetic["Synthetic research"]
  node_synthetic_model["Synthetic model<br/>[synthetic_model.py]"]
  node_activation_generator["Activation generator"]
  node_correlation["Feature correlation<br/>[correlation.py]"]
  node_feature_dictionary["Feature dictionary"]
  node_hierarchy["Feature hierarchy<br/>[hierarchy.py]"]
  node_hierarchy_modifier["Hierarchy modifiers<br/>[modifier.py]"]
  node_hierarchy_node["Hierarchy nodes<br/>[node.py]"]
  node_synthetic_eval["Synthetic SAE evaluation<br/>[evals.py]"]
end

node_researcher(("Researcher"))

node_researcher -->|"configures run"| node_config
node_researcher -->|"loads pretrained SAE"| node_pretrained_load
node_config -->|"configures"| node_train_runner
node_config -->|"configures"| node_cache_runner
node_config -->|"configures"| node_pretokenize
node_pretokenize -->|"uses"| node_token_batch
node_model_load -->|"loads model for"| node_train_runner
node_train_runner -->|"creates data provider"| node_activation_store
node_cache_runner -->|"uses"| node_activation_store
node_activation_store -->|"uses"| node_token_batch
node_activation_store -->|"extracts activations"| node_model_load
node_train_runner -->|"runs training"| node_trainer
node_train_runner -->|"uses"| node_scaler
node_train_runner -->|"evaluates SAE"| node_evals
node_multi_runner -->|"runs training"| node_multi_trainer
node_multi_runner -->|"evaluates SAE"| node_evals
node_trainer -->|"trains"| node_sae
node_trainer -->|"scales activations"| node_scaler
node_multi_trainer -->|"delegates to"| node_trainer
node_multi_trainer -->|"uses"| node_scaler
node_sae -->|"registers architectures"| node_registry
node_pretrained_load -->|"selects architecture"| node_registry
node_pretrained_load -->|"reads catalog"| node_pretrained_catalog
node_pretrained_load -->|"constructs SAE"| node_sae
node_evals -->|"reads activations"| node_activation_store
node_evals -->|"evaluates"| node_sae
node_evals -->|"uses"| node_scaler
node_hooked -->|"integrates SAE"| node_sae
node_bridge -->|"uses"| node_hooked
node_researcher -->|"analyzes with"| node_hooked
node_researcher -->|"explores features"| node_neuronpedia
node_synthetic_model -->|"uses"| node_activation_generator
node_synthetic_model -->|"uses"| node_correlation
node_synthetic_model -->|"uses"| node_feature_dictionary
node_hierarchy -->|"uses"| node_hierarchy_modifier
node_hierarchy -->|"uses"| node_hierarchy_node
node_hierarchy_modifier -->|"modifies nodes"| node_hierarchy_node
node_activation_generator -->|"uses"| node_correlation
node_synthetic_eval -->|"evaluates against"| node_activation_generator
node_synthetic_eval -->|"uses"| node_feature_dictionary
node_researcher -->|"generates data"| node_synthetic_model

click node_config "https://github.com/jbloomaus/saelens/blob/main/sae_lens/config.py"
click node_model_load "https://github.com/jbloomaus/saelens/blob/main/sae_lens/load_model.py"
click node_pretokenize "https://github.com/jbloomaus/saelens/blob/main/sae_lens/pretokenize_runner.py"
click node_token_batch "https://github.com/jbloomaus/saelens/blob/main/sae_lens/tokenization_and_batching.py"
click node_cache_runner "https://github.com/jbloomaus/saelens/blob/main/sae_lens/cache_activations_runner.py"
click node_activation_store "https://github.com/jbloomaus/saelens/blob/main/sae_lens/training/activations_store.py"
click node_scaler "https://github.com/jbloomaus/saelens/blob/main/sae_lens/training/activation_scaler.py"
click node_train_runner "https://github.com/jbloomaus/saelens/blob/main/sae_lens/llm_sae_training_runner.py"
click node_multi_runner "https://github.com/jbloomaus/saelens/blob/main/sae_lens/multi_sae_training_runner.py"
click node_trainer "https://github.com/jbloomaus/saelens/blob/main/sae_lens/training/sae_trainer.py"
click node_multi_trainer "https://github.com/jbloomaus/saelens/blob/main/sae_lens/training/multi_sae_trainer.py"
click node_sae "https://github.com/jbloomaus/saelens/blob/main/sae_lens/saes/sae.py"
click node_registry "https://github.com/jbloomaus/saelens/blob/main/sae_lens/registry.py"
click node_pretrained_load "https://github.com/jbloomaus/saelens/blob/main/sae_lens/loading/pretrained_sae_loaders.py"
click node_pretrained_catalog "https://github.com/jbloomaus/saelens/blob/main/sae_lens/loading/pretrained_saes_directory.py"
click node_evals "https://github.com/jbloomaus/saelens/blob/main/sae_lens/evals.py"
click node_hooked "https://github.com/jbloomaus/saelens/blob/main/sae_lens/analysis/hooked_sae_transformer.py"
click node_bridge "https://github.com/jbloomaus/saelens/blob/main/sae_lens/analysis/sae_transformer_bridge.py"
click node_neuronpedia "https://github.com/jbloomaus/saelens/blob/main/sae_lens/analysis/neuronpedia_integration.py"
click node_synthetic_model "https://github.com/jbloomaus/saelens/blob/main/sae_lens/synthetic/synthetic_model.py"
click node_activation_generator "https://github.com/jbloomaus/saelens/blob/main/sae_lens/synthetic/activation_generator.py"
click node_correlation "https://github.com/jbloomaus/saelens/blob/main/sae_lens/synthetic/correlation.py"
click node_feature_dictionary "https://github.com/jbloomaus/saelens/blob/main/sae_lens/synthetic/feature_dictionary.py"
click node_hierarchy "https://github.com/jbloomaus/saelens/blob/main/sae_lens/synthetic/hierarchy/hierarchy.py"
click node_hierarchy_modifier "https://github.com/jbloomaus/saelens/blob/main/sae_lens/synthetic/hierarchy/modifier.py"
click node_hierarchy_node "https://github.com/jbloomaus/saelens/blob/main/sae_lens/synthetic/hierarchy/node.py"
click node_synthetic_eval "https://github.com/jbloomaus/saelens/blob/main/sae_lens/synthetic/evals.py"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_config,node_model_load,node_pretokenize,node_token_batch,node_cache_runner,node_activation_store,node_scaler,node_train_runner,node_multi_runner,node_trainer,node_multi_trainer toneBlue
class node_sae,node_registry,node_pretrained_load,node_pretrained_catalog toneAmber
class node_evals,node_hooked,node_bridge,node_neuronpedia toneMint
class node_synthetic_model,node_activation_generator,node_correlation,node_feature_dictionary,node_hierarchy,node_hierarchy_modifier,node_hierarchy_node,node_synthetic_eval toneRose
class node_researcher toneIndigo
```

## Components

### SAE training

- [Run configuration](https://github.com/jbloomaus/saelens/blob/main/sae_lens/config.py): `sae_lens/config.py`
- [Language model loading](https://github.com/jbloomaus/saelens/blob/main/sae_lens/load_model.py): `sae_lens/load_model.py`
- [Dataset pretokenization](https://github.com/jbloomaus/saelens/blob/main/sae_lens/pretokenize_runner.py): `sae_lens/pretokenize_runner.py`
- [Tokenization and batching](https://github.com/jbloomaus/saelens/blob/main/sae_lens/tokenization_and_batching.py): `sae_lens/tokenization_and_batching.py`
- [Activation caching](https://github.com/jbloomaus/saelens/blob/main/sae_lens/cache_activations_runner.py): `sae_lens/cache_activations_runner.py`
- [Activation store](https://github.com/jbloomaus/saelens/blob/main/sae_lens/training/activations_store.py): `sae_lens/training/activations_store.py`
- [Activation scaling](https://github.com/jbloomaus/saelens/blob/main/sae_lens/training/activation_scaler.py): `sae_lens/training/activation_scaler.py`
- [Language-model training runner](https://github.com/jbloomaus/saelens/blob/main/sae_lens/llm_sae_training_runner.py): `sae_lens/llm_sae_training_runner.py`
- [Multi-SAE training runner](https://github.com/jbloomaus/saelens/blob/main/sae_lens/multi_sae_training_runner.py): `sae_lens/multi_sae_training_runner.py`
- [SAE trainer](https://github.com/jbloomaus/saelens/blob/main/sae_lens/training/sae_trainer.py): `sae_lens/training/sae_trainer.py`
- [Multi-SAE trainer](https://github.com/jbloomaus/saelens/blob/main/sae_lens/training/multi_sae_trainer.py): `sae_lens/training/multi_sae_trainer.py`

### SAE models and loading

- [SAE architectures](https://github.com/jbloomaus/saelens/blob/main/sae_lens/saes/sae.py): `sae_lens/saes/sae.py`
- [Architecture registry](https://github.com/jbloomaus/saelens/blob/main/sae_lens/registry.py): `sae_lens/registry.py`
- [Pretrained SAE loading](https://github.com/jbloomaus/saelens/blob/main/sae_lens/loading/pretrained_sae_loaders.py): `sae_lens/loading/pretrained_sae_loaders.py`
- [Pretrained SAE catalog](https://github.com/jbloomaus/saelens/blob/main/sae_lens/loading/pretrained_saes_directory.py): `sae_lens/loading/pretrained_saes_directory.py`

### Inference and analysis

- [SAE evaluation](https://github.com/jbloomaus/saelens/blob/main/sae_lens/evals.py): `sae_lens/evals.py`
- [Hooked SAE transformer](https://github.com/jbloomaus/saelens/blob/main/sae_lens/analysis/hooked_sae_transformer.py): `sae_lens/analysis/hooked_sae_transformer.py`
- [Transformer bridge](https://github.com/jbloomaus/saelens/blob/main/sae_lens/analysis/sae_transformer_bridge.py): `sae_lens/analysis/sae_transformer_bridge.py`
- [Neuronpedia integration](https://github.com/jbloomaus/saelens/blob/main/sae_lens/analysis/neuronpedia_integration.py): `sae_lens/analysis/neuronpedia_integration.py`

### Synthetic research

- [Synthetic model](https://github.com/jbloomaus/saelens/blob/main/sae_lens/synthetic/synthetic_model.py): `sae_lens/synthetic/synthetic_model.py`
- [Activation generator](https://github.com/jbloomaus/saelens/blob/main/sae_lens/synthetic/activation_generator.py): `sae_lens/synthetic/activation_generator.py`
- [Feature correlation](https://github.com/jbloomaus/saelens/blob/main/sae_lens/synthetic/correlation.py): `sae_lens/synthetic/correlation.py`
- [Feature dictionary](https://github.com/jbloomaus/saelens/blob/main/sae_lens/synthetic/feature_dictionary.py): `sae_lens/synthetic/feature_dictionary.py`
- [Feature hierarchy](https://github.com/jbloomaus/saelens/blob/main/sae_lens/synthetic/hierarchy/hierarchy.py): `sae_lens/synthetic/hierarchy/hierarchy.py`
- [Hierarchy modifiers](https://github.com/jbloomaus/saelens/blob/main/sae_lens/synthetic/hierarchy/modifier.py): `sae_lens/synthetic/hierarchy/modifier.py`
- [Hierarchy nodes](https://github.com/jbloomaus/saelens/blob/main/sae_lens/synthetic/hierarchy/node.py): `sae_lens/synthetic/hierarchy/node.py`
- [Synthetic SAE evaluation](https://github.com/jbloomaus/saelens/blob/main/sae_lens/synthetic/evals.py): `sae_lens/synthetic/evals.py`

### Other components

- **Researcher**

## Connections

- Researcher → Run configuration: configures run (evidence: `README.md`)
- Researcher → Pretrained SAE loading: loads pretrained SAE (evidence: `README.md`)
- Run configuration → Language-model training runner: configures (evidence: `sae_lens/__init__.py`)
- Run configuration → Activation caching: configures (evidence: `sae_lens/cache_activations_runner.py`)
- Run configuration → Dataset pretokenization: configures (evidence: `sae_lens/pretokenize_runner.py`)
- Dataset pretokenization → Tokenization and batching: uses (evidence: `sae_lens/pretokenize_runner.py`)
- Language model loading → Language-model training runner: loads model for (evidence: `sae_lens/llm_sae_training_runner.py`)
- Language-model training runner → Activation store: creates data provider (evidence: `sae_lens/llm_sae_training_runner.py`)
- Activation caching → Activation store: uses (evidence: `sae_lens/cache_activations_runner.py`)
- Activation store → Tokenization and batching: uses (evidence: `sae_lens/training/activations_store.py`)
- Activation store → Language model loading: extracts activations (evidence: `sae_lens/training/activations_store.py`)
- Language-model training runner → SAE trainer: runs training (evidence: `sae_lens/llm_sae_training_runner.py`)
- Language-model training runner → Activation scaling: uses (evidence: `sae_lens/llm_sae_training_runner.py`)
- Language-model training runner → SAE evaluation: evaluates SAE (evidence: `sae_lens/llm_sae_training_runner.py`)
- Multi-SAE training runner → Multi-SAE trainer: runs training (evidence: `sae_lens/multi_sae_training_runner.py`)
- Multi-SAE training runner → SAE evaluation: evaluates SAE (evidence: `sae_lens/multi_sae_training_runner.py`)
- SAE trainer → SAE architectures: trains (evidence: `sae_lens/training/sae_trainer.py`)
- SAE trainer → Activation scaling: scales activations (evidence: `sae_lens/training/sae_trainer.py`)
- Multi-SAE trainer → SAE trainer: delegates to (evidence: `sae_lens/training/multi_sae_trainer.py`)
- Multi-SAE trainer → Activation scaling: uses (evidence: `sae_lens/training/multi_sae_trainer.py`)
- SAE architectures → Architecture registry: registers architectures (evidence: `sae_lens/registry.py`)
- Pretrained SAE loading → Architecture registry: selects architecture (evidence: `sae_lens/loading/pretrained_sae_loaders.py`)
- Pretrained SAE loading → Pretrained SAE catalog: reads catalog (evidence: `sae_lens/loading/pretrained_sae_loaders.py`)
- Pretrained SAE loading → SAE architectures: constructs SAE (evidence: `sae_lens/saes/sae.py`)
- SAE evaluation → Activation store: reads activations (evidence: `sae_lens/evals.py`)
- SAE evaluation → SAE architectures: evaluates (evidence: `sae_lens/evals.py`)
- SAE evaluation → Activation scaling: uses (evidence: `sae_lens/evals.py`)
- Hooked SAE transformer → SAE architectures: integrates SAE (evidence: `sae_lens/analysis/hooked_sae_transformer.py`)
- Transformer bridge → Hooked SAE transformer: uses (evidence: `sae_lens/analysis/sae_transformer_bridge.py`)
- Researcher → Hooked SAE transformer: analyzes with (evidence: `README.md`)
- Researcher → Neuronpedia integration: explores features (evidence: `README.md`)
- Synthetic model → Activation generator: uses (evidence: `sae_lens/synthetic/synthetic_model.py`)
- Synthetic model → Feature correlation: uses (evidence: `sae_lens/synthetic/synthetic_model.py`)
- Synthetic model → Feature dictionary: uses (evidence: `sae_lens/synthetic/synthetic_model.py`)
- Feature hierarchy → Hierarchy modifiers: uses (evidence: `sae_lens/synthetic/hierarchy/hierarchy.py`)
- Feature hierarchy → Hierarchy nodes: uses (evidence: `sae_lens/synthetic/hierarchy/hierarchy.py`)
- Hierarchy modifiers → Hierarchy nodes: modifies nodes (evidence: `sae_lens/synthetic/hierarchy/modifier.py`)
- Activation generator → Feature correlation: uses (evidence: `sae_lens/synthetic/activation_generator.py`)
- Synthetic SAE evaluation → Activation generator: evaluates against (evidence: `sae_lens/synthetic/evals.py`)
- Synthetic SAE evaluation → Feature dictionary: uses (evidence: `sae_lens/synthetic/evals.py`)
- Researcher → Synthetic model: generates data (evidence: `README.md`)
