# psakthivel353/sae-feature-visualizer architecture

> Architecture diagram of the GitHub repository psakthivel353/sae-feature-visualizer, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-10-02). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/psakthivel353/sae-feature-visualizer
- Repository: https://github.com/psakthivel353/sae-feature-visualizer

## Overview

SAE Feature Visualizer is a research workflow for turning GPT-2 layer activations into sparse features, corpus-based interpretations, and interactive exploration. Users prepare a text corpus, collect hooked activations, train a Top-K sparse autoencoder, analyze feature activations and examples, then inspect results in the Streamlit dashboard; feature labels are an optional manual enrichment. The graph includes README-documented capabilities whose implementation was not fully excerpted, while omitting uncertain internal links where source evidence was unavailable.

Main entry points: [`src/dashboard.py`](https://github.com/psakthivel353/sae-feature-visualizer/blob/main/src/dashboard.py) (Streamlit Dashboard), [`src/collect_activations.py`](https://github.com/psakthivel353/sae-feature-visualizer/blob/main/src/collect_activations.py) (Activation Collection), [`src/feature_analysis.py`](https://github.com/psakthivel353/sae-feature-visualizer/blob/main/src/feature_analysis.py) (Feature Analysis), [`cache/layer8_chunk_manifest.json`](https://github.com/psakthivel353/sae-feature-visualizer/blob/main/cache/layer8_chunk_manifest.json) (Activation Cache)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_data["Corpus and Activations"]
  node_corpus_prep["Corpus Preparation<br/>[download_corpus.py]"]
  node_corpus["Text Corpus"]
  node_collector["Activation Collection"]
  node_hooks["Forward Hooks<br/>[hooks.py]"]
  node_activation_cache[("Activation Cache")]
end

subgraph group_model["Sparse Modeling"]
  node_trainer["SAE Training<br/>[train_sae.py]"]
  node_sae["Sparse Autoencoder<br/>[sae_model.py]"]
  node_checkpoints[("Model Checkpoints")]
end

subgraph group_interpret["Feature Interpretation"]
  node_analyzer["Feature Analysis"]
  node_feature_results[("Feature Statistics and Examples<br/>[feature_stats.json]")]
  node_labeler["Manual Feature Labeling<br/>[label_features.py]"]
  node_feature_labels["Feature Labels"]
end

subgraph group_product["Interactive Exploration"]
  node_dashboard["Streamlit Dashboard<br/>[dashboard.py]"]
end

node_user(("Researcher"))
node_datasets(("Corpus Providers"))
node_gpt2["GPT-2"]

node_datasets -->|"provides text"| node_corpus_prep
node_corpus_prep -->|"writes corpus"| node_corpus
node_corpus -->|"supplies text"| node_collector
node_collector -->|"runs inference"| node_gpt2
node_collector -->|"attaches hooks"| node_hooks
node_hooks -->|"captures layer output"| node_gpt2
node_collector -->|"writes chunks"| node_activation_cache
node_activation_cache -->|"supplies activations"| node_trainer
node_trainer -->|"trains model"| node_sae
node_trainer -->|"saves weights"| node_checkpoints
node_activation_cache -->|"provides activations"| node_analyzer
node_checkpoints -->|"loads model"| node_analyzer
node_analyzer -->|"encodes activations"| node_sae
node_analyzer -->|"produces statistics and examples"| node_feature_results
node_feature_results -->|"guides labeling"| node_labeler
node_labeler -->|"saves labels"| node_feature_labels
node_user -->|"explores features"| node_dashboard
node_feature_results -->|"displays results"| node_dashboard
node_checkpoints -->|"loads checkpoint"| node_dashboard
node_feature_labels -.->|"shows labels"| node_dashboard
node_dashboard -.->|"runs live inference"| node_gpt2
node_dashboard -->|"uses activation hooks"| node_hooks
node_dashboard -->|"encodes live activations"| node_sae

click node_corpus_prep "https://github.com/psakthivel353/sae-feature-visualizer/blob/main/scripts/download_corpus.py"
click node_corpus "https://github.com/psakthivel353/sae-feature-visualizer/tree/main/data"
click node_collector "https://github.com/psakthivel353/sae-feature-visualizer/blob/main/src/collect_activations.py"
click node_hooks "https://github.com/psakthivel353/sae-feature-visualizer/blob/main/src/hooks.py"
click node_activation_cache "https://github.com/psakthivel353/sae-feature-visualizer/blob/main/cache/layer8_chunk_manifest.json"
click node_trainer "https://github.com/psakthivel353/sae-feature-visualizer/blob/main/src/train_sae.py"
click node_sae "https://github.com/psakthivel353/sae-feature-visualizer/blob/main/src/sae_model.py"
click node_checkpoints "https://github.com/psakthivel353/sae-feature-visualizer/tree/main/checkpoints"
click node_analyzer "https://github.com/psakthivel353/sae-feature-visualizer/blob/main/src/feature_analysis.py"
click node_feature_results "https://github.com/psakthivel353/sae-feature-visualizer/blob/main/cache/feature_stats.json"
click node_labeler "https://github.com/psakthivel353/sae-feature-visualizer/blob/main/scripts/label_features.py"
click node_feature_labels "https://github.com/psakthivel353/sae-feature-visualizer/tree/main/cache"
click node_dashboard "https://github.com/psakthivel353/sae-feature-visualizer/blob/main/src/dashboard.py"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_corpus_prep,node_corpus,node_collector,node_hooks,node_activation_cache toneBlue
class node_trainer,node_sae,node_checkpoints toneAmber
class node_analyzer,node_feature_results,node_labeler,node_feature_labels toneMint
class node_dashboard toneRose
class node_user,node_datasets,node_gpt2 toneIndigo
```

## Components

### Corpus and Activations

- [Corpus Preparation](https://github.com/psakthivel353/sae-feature-visualizer/blob/main/scripts/download_corpus.py): `scripts/download_corpus.py`
- [Text Corpus](https://github.com/psakthivel353/sae-feature-visualizer/tree/main/data): `data`
- [Activation Collection](https://github.com/psakthivel353/sae-feature-visualizer/blob/main/src/collect_activations.py): `src/collect_activations.py`
- [Forward Hooks](https://github.com/psakthivel353/sae-feature-visualizer/blob/main/src/hooks.py): `src/hooks.py`
- [Activation Cache](https://github.com/psakthivel353/sae-feature-visualizer/blob/main/cache/layer8_chunk_manifest.json): `cache/layer8_chunk_manifest.json`

### Sparse Modeling

- [SAE Training](https://github.com/psakthivel353/sae-feature-visualizer/blob/main/src/train_sae.py): `src/train_sae.py`
- [Sparse Autoencoder](https://github.com/psakthivel353/sae-feature-visualizer/blob/main/src/sae_model.py): `src/sae_model.py`
- [Model Checkpoints](https://github.com/psakthivel353/sae-feature-visualizer/tree/main/checkpoints): `checkpoints`

### Feature Interpretation

- [Feature Analysis](https://github.com/psakthivel353/sae-feature-visualizer/blob/main/src/feature_analysis.py): `src/feature_analysis.py`
- [Feature Statistics and Examples](https://github.com/psakthivel353/sae-feature-visualizer/blob/main/cache/feature_stats.json): `cache/feature_stats.json`
- [Manual Feature Labeling](https://github.com/psakthivel353/sae-feature-visualizer/blob/main/scripts/label_features.py): `scripts/label_features.py`
- [Feature Labels](https://github.com/psakthivel353/sae-feature-visualizer/tree/main/cache): `cache`

### Interactive Exploration

- [Streamlit Dashboard](https://github.com/psakthivel353/sae-feature-visualizer/blob/main/src/dashboard.py): `src/dashboard.py`

### Other components

- **Researcher**
- **Corpus Providers**
- **GPT-2**

## Connections

- Corpus Providers → Corpus Preparation: provides text (evidence: `scripts/download_corpus.py`)
- Corpus Preparation → Text Corpus: writes corpus (evidence: `scripts/download_corpus.py`)
- Text Corpus → Activation Collection: supplies text (evidence: `src/collect_activations.py`)
- Activation Collection → GPT-2: runs inference (evidence: `src/collect_activations.py`)
- Activation Collection → Forward Hooks: attaches hooks (evidence: `src/collect_activations.py`)
- Forward Hooks → GPT-2: captures layer output (evidence: `src/hooks.py`)
- Activation Collection → Activation Cache: writes chunks (evidence: `src/collect_activations.py`)
- Activation Cache → SAE Training: supplies activations (evidence: `src/train_sae.py`)
- SAE Training → Sparse Autoencoder: trains model (evidence: `src/train_sae.py`)
- SAE Training → Model Checkpoints: saves weights (evidence: `src/train_sae.py`)
- Activation Cache → Feature Analysis: provides activations (evidence: `src/feature_analysis.py`)
- Model Checkpoints → Feature Analysis: loads model (evidence: `src/feature_analysis.py`)
- Feature Analysis → Sparse Autoencoder: encodes activations (evidence: `src/feature_analysis.py`)
- Feature Analysis → Feature Statistics and Examples: produces statistics and examples (evidence: `src/feature_analysis.py`)
- Feature Statistics and Examples → Manual Feature Labeling: guides labeling (evidence: `scripts/label_features.py`)
- Manual Feature Labeling → Feature Labels: saves labels (evidence: `scripts/label_features.py`)
- Researcher → Streamlit Dashboard: explores features (evidence: `README.md`)
- Feature Statistics and Examples → Streamlit Dashboard: displays results (evidence: `src/dashboard.py`)
- Model Checkpoints → Streamlit Dashboard: loads checkpoint (evidence: `src/dashboard.py`)
- Feature Labels → Streamlit Dashboard: shows labels
- Streamlit Dashboard → GPT-2: runs live inference (evidence: `README.md`)
- Streamlit Dashboard → Forward Hooks: uses activation hooks (evidence: `src/dashboard.py`)
- Streamlit Dashboard → Sparse Autoencoder: encodes live activations (evidence: `src/dashboard.py`)
