# andyrdt/refusal_direction architecture

> Architecture diagram of the GitHub repository andyrdt/refusal_direction, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-09-27). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/andyrdt/refusal_direction
- Repository: https://github.com/andyrdt/refusal_direction

## Overview

This repository reproduces research on refusal directions in language models. The main workflow loads harmful and harmless prompt data and a selected model, extracts candidate directions, selects one using refusal, steering, and KL metrics, then evaluates completions and cross-entropy loss with and without interventions. Model-family adapters and activation hooks provide the internal layer for running the same experiments across architectures. The README documents the stages; sampled source confirms dataset loading, orchestration helpers, evaluation logic, and hook behavior, but not every call site.

Main entry points: [`pipeline/run_pipeline.py`](https://github.com/andyrdt/refusal_direction/blob/main/pipeline/run_pipeline.py) (Pipeline Runner), [`pipeline/runs`](https://github.com/andyrdt/refusal_direction/tree/main/pipeline/runs) (Experiment Artifacts), [`pipeline/submodules/select_direction.py`](https://github.com/andyrdt/refusal_direction/blob/main/pipeline/submodules/select_direction.py) (Direction Selection), [`dataset/load_dataset.py`](https://github.com/andyrdt/refusal_direction/blob/main/dataset/load_dataset.py) (Dataset Loader)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_data["Prompt Data"]
  node_loader["Dataset Loader<br/>[load_dataset.py]"]
  node_promptsets["Prompt Datasets"]
  node_benchmarksets["Evaluation Sets"]
end

subgraph group_workflow["Experiment Workflow"]
  node_runner["Pipeline Runner<br/>[run_pipeline.py]"]
  node_config["Run Configuration<br/>[config.py]"]
  node_directiongen["Candidate Directions"]
  node_directionselect["Direction Selection"]
  node_selected["Selected Direction"]
end

subgraph group_model["Model Interventions"]
  node_factory["Model Factory<br/>[model_factory.py]"]
  node_adapters["Model Adapters"]
  node_hooks["Activation Hooks<br/>[hook_utils.py]"]
end

subgraph group_evaluation["Evaluation"]
  node_completiongen["Completion Generation<br/>[run_pipeline.py]"]
  node_completioneval["Safety Evaluation"]
  node_losseval["Loss Evaluation<br/>[evaluate_loss.py]"]
  node_artifacts["Experiment Artifacts"]
end

node_researcher(("Researcher"))
node_hf["Hugging Face Models"]
node_judge["Together AI Judge"]

node_researcher -->|"starts run"| node_runner
node_runner -->|"configures"| node_config
node_runner -->|"loads prompts"| node_loader
node_loader -->|"reads splits"| node_promptsets
node_loader -->|"reads datasets"| node_benchmarksets
node_runner -->|"creates model"| node_factory
node_factory -->|"selects adapter"| node_adapters
node_adapters -->|"loads weights"| node_hf
node_runner -->|"extracts candidates"| node_directiongen
node_directiongen -->|"measures activations"| node_adapters
node_runner -->|"selects direction"| node_directionselect
node_directionselect -->|"tests interventions"| node_hooks
node_directionselect -->|"saves direction"| node_selected
node_runner -->|"generates outputs"| node_completiongen
node_completiongen -->|"generates completions"| node_adapters
node_completiongen -->|"applies hooks"| node_hooks
node_completiongen -->|"loads evaluation set"| node_loader
node_runner -->|"evaluates completions"| node_completioneval
node_completioneval -.->|"requests classifications"| node_judge
node_runner -->|"evaluates loss"| node_losseval
node_losseval -->|"applies hooks"| node_hooks
node_runner -->|"writes results"| node_artifacts
node_directiongen -->|"writes candidates"| node_artifacts
node_directionselect -->|"writes metrics"| node_artifacts
node_completiongen -->|"writes completions"| node_artifacts
node_completioneval -->|"writes evaluations"| node_artifacts
node_losseval -->|"writes loss metrics"| node_artifacts

click node_runner "https://github.com/andyrdt/refusal_direction/blob/main/pipeline/run_pipeline.py"
click node_config "https://github.com/andyrdt/refusal_direction/blob/main/pipeline/config.py"
click node_loader "https://github.com/andyrdt/refusal_direction/blob/main/dataset/load_dataset.py"
click node_promptsets "https://github.com/andyrdt/refusal_direction/tree/main/dataset/splits"
click node_benchmarksets "https://github.com/andyrdt/refusal_direction/tree/main/dataset/processed"
click node_factory "https://github.com/andyrdt/refusal_direction/blob/main/pipeline/model_utils/model_factory.py"
click node_adapters "https://github.com/andyrdt/refusal_direction/tree/main/pipeline/model_utils"
click node_hooks "https://github.com/andyrdt/refusal_direction/blob/main/pipeline/utils/hook_utils.py"
click node_directiongen "https://github.com/andyrdt/refusal_direction/blob/main/pipeline/submodules/generate_directions.py"
click node_directionselect "https://github.com/andyrdt/refusal_direction/blob/main/pipeline/submodules/select_direction.py"
click node_selected "https://github.com/andyrdt/refusal_direction/tree/main/pipeline/runs"
click node_completiongen "https://github.com/andyrdt/refusal_direction/blob/main/pipeline/run_pipeline.py"
click node_completioneval "https://github.com/andyrdt/refusal_direction/blob/main/pipeline/submodules/evaluate_jailbreak.py"
click node_losseval "https://github.com/andyrdt/refusal_direction/blob/main/pipeline/submodules/evaluate_loss.py"
click node_artifacts "https://github.com/andyrdt/refusal_direction/tree/main/pipeline/runs"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_loader,node_promptsets,node_benchmarksets toneBlue
class node_runner,node_config,node_directiongen,node_directionselect,node_selected toneAmber
class node_factory,node_adapters,node_hooks toneMint
class node_completiongen,node_completioneval,node_losseval,node_artifacts toneRose
class node_researcher,node_hf,node_judge toneIndigo
```

## Components

### Prompt Data

- [Dataset Loader](https://github.com/andyrdt/refusal_direction/blob/main/dataset/load_dataset.py): `dataset/load_dataset.py`
- [Prompt Datasets](https://github.com/andyrdt/refusal_direction/tree/main/dataset/splits): `dataset/splits`
- [Evaluation Sets](https://github.com/andyrdt/refusal_direction/tree/main/dataset/processed): `dataset/processed`

### Experiment Workflow

- [Pipeline Runner](https://github.com/andyrdt/refusal_direction/blob/main/pipeline/run_pipeline.py): `pipeline/run_pipeline.py`
- [Run Configuration](https://github.com/andyrdt/refusal_direction/blob/main/pipeline/config.py): `pipeline/config.py`
- [Candidate Directions](https://github.com/andyrdt/refusal_direction/blob/main/pipeline/submodules/generate_directions.py): `pipeline/submodules/generate_directions.py`
- [Direction Selection](https://github.com/andyrdt/refusal_direction/blob/main/pipeline/submodules/select_direction.py): `pipeline/submodules/select_direction.py`
- [Selected Direction](https://github.com/andyrdt/refusal_direction/tree/main/pipeline/runs): `pipeline/runs`

### Model Interventions

- [Model Factory](https://github.com/andyrdt/refusal_direction/blob/main/pipeline/model_utils/model_factory.py): `pipeline/model_utils/model_factory.py`
- [Model Adapters](https://github.com/andyrdt/refusal_direction/tree/main/pipeline/model_utils): `pipeline/model_utils`
- [Activation Hooks](https://github.com/andyrdt/refusal_direction/blob/main/pipeline/utils/hook_utils.py): `pipeline/utils/hook_utils.py`

### Evaluation

- [Completion Generation](https://github.com/andyrdt/refusal_direction/blob/main/pipeline/run_pipeline.py): `pipeline/run_pipeline.py`
- [Safety Evaluation](https://github.com/andyrdt/refusal_direction/blob/main/pipeline/submodules/evaluate_jailbreak.py): `pipeline/submodules/evaluate_jailbreak.py`
- [Loss Evaluation](https://github.com/andyrdt/refusal_direction/blob/main/pipeline/submodules/evaluate_loss.py): `pipeline/submodules/evaluate_loss.py`
- [Experiment Artifacts](https://github.com/andyrdt/refusal_direction/tree/main/pipeline/runs): `pipeline/runs`

### Other components

- **Researcher**
- **Hugging Face Models**
- **Together AI Judge**

## Connections

- Researcher → Pipeline Runner: starts run
- Pipeline Runner → Run Configuration: configures
- Pipeline Runner → Dataset Loader: loads prompts
- Dataset Loader → Prompt Datasets: reads splits
- Dataset Loader → Evaluation Sets: reads datasets
- Pipeline Runner → Model Factory: creates model
- Model Factory → Model Adapters: selects adapter
- Model Adapters → Hugging Face Models: loads weights
- Pipeline Runner → Candidate Directions: extracts candidates
- Candidate Directions → Model Adapters: measures activations
- Pipeline Runner → Direction Selection: selects direction
- Direction Selection → Activation Hooks: tests interventions
- Direction Selection → Selected Direction: saves direction
- Pipeline Runner → Completion Generation: generates outputs
- Completion Generation → Model Adapters: generates completions
- Completion Generation → Activation Hooks: applies hooks
- Completion Generation → Dataset Loader: loads evaluation set
- Pipeline Runner → Safety Evaluation: evaluates completions
- Safety Evaluation → Together AI Judge: requests classifications
- Pipeline Runner → Loss Evaluation: evaluates loss
- Loss Evaluation → Activation Hooks: applies hooks
- Pipeline Runner → Experiment Artifacts: writes results
- Candidate Directions → Experiment Artifacts: writes candidates
- Direction Selection → Experiment Artifacts: writes metrics
- Completion Generation → Experiment Artifacts: writes completions
- Safety Evaluation → Experiment Artifacts: writes evaluations
- Loss Evaluation → Experiment Artifacts: writes loss metrics