How jacobdunefsky/transcoder_circuits works
Project Type and Purpose
• This is an open-source research toolkit for reverse-engineering transformer MLP sublayers in large language models using “transcoders.”
• It provides:
• Core libraries for training and applying sparse autoencoders (“SAEs”) and transcoders (sae_training/)
• A set of analysis tools for interpreting circuits (transcoder_circuits/)
• Example scripts and Jupyter notebooks (walkthrough.ipynb, case studies, sweep, interp-comparison)
• Installation and dependency management (setup.sh, requirements.txt)
High-Level Components
• Installation Layer
• setup.sh, requirements.txt → installs Python dependencies, downloads pretrained transcoder weights from HuggingFace
• Core Library Layer
• sae_training/ → modules for training SAE/transcoder models, configuration, utilities, a nested geom_median package
• transcoder_circuits/ → modules for analyzing circuits, generating feature dashboards, replacement-context experiments
• Example and Experiment Layer
• train_transcoder.py → script demonstrating transcoder training
• sweep.ipynb, interp-comparison.ipynb → notebooks for large-scale evaluation
• walkthrough.ipynb → guided tutorial of core workflows
• case_study_*.ipynb → deep-dive analyses of specific transcoder features
• Data and Artifacts
• Downloaded weights (HuggingFace)
• Generated activation stores (sae_training/activations_store.py)
• Outputs of notebooks (figures, logs, dashboards)
Relationships and Data Flow
1. The user runs setup.sh → installs dependencies and pulls pretrained weights.
2. Core libraries (sae_training) consume model weights → train or load transcoders.
3. Analysis library (transcoder_circuits) takes trained transcoders + model activations → computes interpreter dashboards, feature analyses.
4. Example scripts and notebooks orchestrate calls into the libraries and produce visualizations or evaluation metrics.
Architectural Patterns and Principles
• Layered design: installation, core libraries, front-end notebooks/scripts
• Clear separation of concerns: training (sae_training) vs. analysis (transcoder_circuits)
• Extensibility:
• geom_median is a plug-in style subpackage for specialized algorithms
• Users can add new notebooks or scripts to call the libraries