How jbloomaus/saelens works
SAE Lens is a research library for training and analyzing sparse autoencoders on language-model activations, and for loading pretrained SAEs. The main workflow configures a run, obtains model activations, trains an SAE, and evaluates or saves it. Separate capabilities support inference-time model integration and synthetic-data research, including hierarchical synthetic features. The graph reflects README-documented capabilities and SOURCE INDEX references; unsampled implementation details are not inferred as call relationships.
Main entry points: llm_sae_training_runner.py Language-model training runner · evals.py SAE evaluation · activations_store.py Activation store · sae.py SAE architectures
Components 28
SAE training
- Run configuration
sae_lens/config.py - Language model loading
sae_lens/load_model.py - Dataset pretokenization
sae_lens/pretokenize_runner.py - Tokenization and batching
sae_lens/tokenization_and_batching.py - Activation caching
sae_lens/cache_activations_runner.py - Activation store
sae_lens/training/activations_store.py - Activation scaling
sae_lens/training/activation_scaler.py - Language-model training runner
sae_lens/llm_sae_training_runner.py - Multi-SAE training runner
sae_lens/multi_sae_training_runner.py - SAE trainer
sae_lens/training/sae_trainer.py - Multi-SAE trainer
sae_lens/training/multi_sae_trainer.py
SAE models and loading
- SAE architectures
sae_lens/saes/sae.py - Architecture registry
sae_lens/registry.py - Pretrained SAE loading
sae_lens/loading/pretrained_sae_loaders.py - Pretrained SAE catalog
sae_lens/loading/pretrained_saes_directory.py
Inference and analysis
- SAE evaluation
sae_lens/evals.py - Hooked SAE transformer
sae_lens/analysis/hooked_sae_transformer.py - Transformer bridge
sae_lens/analysis/sae_transformer_bridge.py - Neuronpedia integration
sae_lens/analysis/neuronpedia_integration.py
Synthetic research
- Synthetic model
sae_lens/synthetic/synthetic_model.py - Activation generator
sae_lens/synthetic/activation_generator.py - Feature correlation
sae_lens/synthetic/correlation.py - Feature dictionary
sae_lens/synthetic/feature_dictionary.py - Feature hierarchy
sae_lens/synthetic/hierarchy/hierarchy.py - Hierarchy modifiers
sae_lens/synthetic/hierarchy/modifier.py - Hierarchy nodes
sae_lens/synthetic/hierarchy/node.py - Synthetic SAE evaluation
sae_lens/synthetic/evals.py
Other components
- Researcher
Connections 41
- Researcher → Run configuration: configures run
README.md - Researcher → Pretrained SAE loading: loads pretrained SAE
README.md - Run configuration → Language-model training runner: configures
sae_lens/__init__.py - Run configuration → Activation caching: configures
sae_lens/cache_activations_runner.py - Run configuration → Dataset pretokenization: configures
sae_lens/pretokenize_runner.py - Dataset pretokenization → Tokenization and batching: uses
sae_lens/pretokenize_runner.py - Language model loading → Language-model training runner: loads model for
sae_lens/llm_sae_training_runner.py - Language-model training runner → Activation store: creates data provider
sae_lens/llm_sae_training_runner.py - Activation caching → Activation store: uses
sae_lens/cache_activations_runner.py - Activation store → Tokenization and batching: uses
sae_lens/training/activations_store.py - Activation store → Language model loading: extracts activations
sae_lens/training/activations_store.py - Language-model training runner → SAE trainer: runs training
sae_lens/llm_sae_training_runner.py - Language-model training runner → Activation scaling: uses
sae_lens/llm_sae_training_runner.py - Language-model training runner → SAE evaluation: evaluates SAE
sae_lens/llm_sae_training_runner.py - Multi-SAE training runner → Multi-SAE trainer: runs training
sae_lens/multi_sae_training_runner.py - Multi-SAE training runner → SAE evaluation: evaluates SAE
sae_lens/multi_sae_training_runner.py - SAE trainer → SAE architectures: trains
sae_lens/training/sae_trainer.py - SAE trainer → Activation scaling: scales activations
sae_lens/training/sae_trainer.py - Multi-SAE trainer → SAE trainer: delegates to
sae_lens/training/multi_sae_trainer.py - Multi-SAE trainer → Activation scaling: uses
sae_lens/training/multi_sae_trainer.py - SAE architectures → Architecture registry: registers architectures
sae_lens/registry.py - Pretrained SAE loading → Architecture registry: selects architecture
sae_lens/loading/pretrained_sae_loaders.py - Pretrained SAE loading → Pretrained SAE catalog: reads catalog
sae_lens/loading/pretrained_sae_loaders.py - Pretrained SAE loading → SAE architectures: constructs SAE
sae_lens/saes/sae.py - SAE evaluation → Activation store: reads activations
sae_lens/evals.py - SAE evaluation → SAE architectures: evaluates
sae_lens/evals.py - SAE evaluation → Activation scaling: uses
sae_lens/evals.py - Hooked SAE transformer → SAE architectures: integrates SAE
sae_lens/analysis/hooked_sae_transformer.py - Transformer bridge → Hooked SAE transformer: uses
sae_lens/analysis/sae_transformer_bridge.py - Researcher → Hooked SAE transformer: analyzes with
README.md - Researcher → Neuronpedia integration: explores features
README.md - Synthetic model → Activation generator: uses
sae_lens/synthetic/synthetic_model.py - Synthetic model → Feature correlation: uses
sae_lens/synthetic/synthetic_model.py - Synthetic model → Feature dictionary: uses
sae_lens/synthetic/synthetic_model.py - Feature hierarchy → Hierarchy modifiers: uses
sae_lens/synthetic/hierarchy/hierarchy.py - Feature hierarchy → Hierarchy nodes: uses
sae_lens/synthetic/hierarchy/hierarchy.py - Hierarchy modifiers → Hierarchy nodes: modifies nodes
sae_lens/synthetic/hierarchy/modifier.py - Activation generator → Feature correlation: uses
sae_lens/synthetic/activation_generator.py - Synthetic SAE evaluation → Activation generator: evaluates against
sae_lens/synthetic/evals.py - Synthetic SAE evaluation → Feature dictionary: uses
sae_lens/synthetic/evals.py - Researcher → Synthetic model: generates data
README.md