# rifoagenadi/sycophancy architecture

> Architecture diagram of the GitHub repository rifoagenadi/sycophancy, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-11-10). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/rifoagenadi/sycophancy
- Repository: https://github.com/rifoagenadi/sycophancy

## Overview

This repository develops and evaluates probes for sycophancy and truthfulness in language-model activations, with a separate dialogue-based steering and uncertainty-quantification workstream. The main workflow prepares labeled TruthfulQA examples, extracts MHA/MLP/residual activations, trains probes, and applies probe directions during inference. Additional branches generate disagreement dialogues, steer or ablate model responses, and evaluate or visualize results. Edges reflect the README or shown source/index references; unsampled subsystem internals and some runtime wiring are intentionally left unspecified.

Main entry points: [`utils.py`](https://github.com/rifoagenadi/sycophancy/blob/main/utils.py) (Model and data loading), [`probe/train.py`](https://github.com/rifoagenadi/sycophancy/blob/main/probe/train.py) (Trained probes), [`extension/metrics.py`](https://github.com/rifoagenadi/sycophancy/blob/main/extension/metrics.py) (Metrics), [`uq/dialogue.py`](https://github.com/rifoagenadi/sycophancy/blob/main/uq/dialogue.py) (Disagreement dialogues)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_probe_workflow["Probe workflow"]
  node_labeled_examples["Labeled examples"]
  node_model_loader["Model and data loading<br/>[utils.py]"]
  node_activation_capture["Activation extraction"]
  node_probe_training["Probe training<br/>[train.py]"]
  node_probe_weights[("Trained probes<br/>[train.py]")]
  node_projection_stats["Projection statistics"]
  node_mha_inference["MHA intervention<br/>[inference_mha.py]"]
  node_mlp_inference["MLP intervention<br/>[inference_mlp.py]"]
  node_residual_inference["Residual intervention"]
end

subgraph group_steering["Dialogue and steering"]
  node_dialogue_generation["Disagreement dialogues<br/>[dialogue.py]"]
  node_extension_dialogues["Steering dialogues<br/>[dialogue.py]"]
  node_extended_activations["Dialogue activations"]
  node_steering_runtime["Steering and ablation<br/>[steer.py]"]
end

subgraph group_evaluation["Evaluation and analysis"]
  node_judge["Response judging<br/>[judge.py]"]
  node_metrics["Metrics<br/>[metrics.py]"]
  node_accuracy_plots["Probe accuracy plots"]
  node_disagreement_plots["Disagreement plots"]
  node_generalization_plots["Steering plots"]
end

subgraph group_uncertainty["Uncertainty quantification"]
  node_uq_dialogue["UQ dialogue<br/>[dialogue.py]"]
  node_uq_judge["UQ judging<br/>[judge.py]"]
  node_uq_label["UQ labeling<br/>[label.py]"]
  node_uq_runner["UQ runner<br/>[run_uqlm.py]"]
  node_uq_visualization["UQ visualization<br/>[visualize.py]"]
end

node_researcher(("Researcher"))
node_truthfulqa[("TruthfulQA data")]
node_model_hub(("Model provider"))
node_responses[("Model responses")]

node_researcher -->|"trains probes"| node_probe_training
node_truthfulqa -->|"provides examples"| node_labeled_examples
node_truthfulqa -->|"loads test data"| node_model_loader
node_model_hub -->|"provides models"| node_model_loader
node_labeled_examples -->|"labels data"| node_probe_training
node_model_loader -->|"loads model"| node_probe_training
node_probe_training -->|"extracts activations"| node_activation_capture
node_probe_training -->|"saves weights"| node_probe_weights
node_probe_weights -->|"projects directions"| node_projection_stats
node_probe_weights -->|"loads directions"| node_mha_inference
node_probe_weights -->|"loads directions"| node_mlp_inference
node_probe_weights -->|"loads directions"| node_residual_inference
node_model_loader -->|"loads model and data"| node_mha_inference
node_model_loader -->|"loads model and data"| node_mlp_inference
node_model_loader -->|"loads model and data"| node_residual_inference
node_mha_inference -->|"writes predictions"| node_responses
node_mlp_inference -->|"writes predictions"| node_responses
node_residual_inference -->|"writes predictions"| node_responses
node_truthfulqa -->|"loads questions"| node_dialogue_generation
node_dialogue_generation -->|"writes dialogues"| node_responses
node_truthfulqa -->|"builds dialogue samples"| node_extended_activations
node_extended_activations -.->|"supplies probe inputs"| node_steering_runtime
node_dialogue_generation -.->|"dialogue reference"| node_steering_runtime
node_researcher -.->|"evaluates responses"| node_judge
node_judge -.->|"provides judgments"| node_metrics
node_metrics -.->|"plots accuracy"| node_accuracy_plots
node_metrics -.->|"plots disagreement"| node_disagreement_plots
node_metrics -.->|"plots generalization"| node_generalization_plots
node_researcher -.->|"runs UQ dialogue"| node_uq_dialogue
node_uq_dialogue -.->|"judgment workflow"| node_uq_judge
node_uq_judge -.->|"labeling workflow"| node_uq_label
node_uq_runner -.->|"analysis workflow"| node_uq_visualization

click node_labeled_examples "https://github.com/rifoagenadi/sycophancy/blob/main/probe/probe_data_utils.py"
click node_model_loader "https://github.com/rifoagenadi/sycophancy/blob/main/utils.py"
click node_activation_capture "https://github.com/rifoagenadi/sycophancy/blob/main/probe/extract_activation.py"
click node_probe_training "https://github.com/rifoagenadi/sycophancy/blob/main/probe/train.py"
click node_probe_weights "https://github.com/rifoagenadi/sycophancy/blob/main/probe/train.py"
click node_projection_stats "https://github.com/rifoagenadi/sycophancy/blob/main/probe/compute_proj_std.py"
click node_mha_inference "https://github.com/rifoagenadi/sycophancy/blob/main/inference_mha.py"
click node_mlp_inference "https://github.com/rifoagenadi/sycophancy/blob/main/inference_mlp.py"
click node_residual_inference "https://github.com/rifoagenadi/sycophancy/blob/main/inference_residual.py"
click node_dialogue_generation "https://github.com/rifoagenadi/sycophancy/blob/main/uq/dialogue.py"
click node_extension_dialogues "https://github.com/rifoagenadi/sycophancy/blob/main/extension/dialogue.py"
click node_extended_activations "https://github.com/rifoagenadi/sycophancy/blob/main/extension/extract_activations.py"
click node_steering_runtime "https://github.com/rifoagenadi/sycophancy/blob/main/extension/steer.py"
click node_judge "https://github.com/rifoagenadi/sycophancy/blob/main/extension/judge.py"
click node_metrics "https://github.com/rifoagenadi/sycophancy/blob/main/extension/metrics.py"
click node_accuracy_plots "https://github.com/rifoagenadi/sycophancy/blob/main/extension/diagram/probe_accuracy_plot.py"
click node_disagreement_plots "https://github.com/rifoagenadi/sycophancy/blob/main/extension/diagram/disagreement_type_plot.py"
click node_generalization_plots "https://github.com/rifoagenadi/sycophancy/blob/main/extension/diagram/steering_generalization_plot.py"
click node_uq_dialogue "https://github.com/rifoagenadi/sycophancy/blob/main/uq/dialogue.py"
click node_uq_judge "https://github.com/rifoagenadi/sycophancy/blob/main/uq/judge.py"
click node_uq_label "https://github.com/rifoagenadi/sycophancy/blob/main/uq/label.py"
click node_uq_runner "https://github.com/rifoagenadi/sycophancy/blob/main/uq/run_uqlm.py"
click node_uq_visualization "https://github.com/rifoagenadi/sycophancy/blob/main/uq/visualize.py"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_labeled_examples,node_model_loader,node_activation_capture,node_probe_training,node_probe_weights,node_projection_stats,node_mha_inference,node_mlp_inference,node_residual_inference toneBlue
class node_dialogue_generation,node_extension_dialogues,node_extended_activations,node_steering_runtime,node_truthfulqa,node_responses toneAmber
class node_judge,node_metrics,node_accuracy_plots,node_disagreement_plots,node_generalization_plots toneMint
class node_uq_dialogue,node_uq_judge,node_uq_label,node_uq_runner,node_uq_visualization toneRose
class node_researcher,node_model_hub toneIndigo
```

## Components

### Probe workflow

- [Labeled examples](https://github.com/rifoagenadi/sycophancy/blob/main/probe/probe_data_utils.py): `probe/probe_data_utils.py`
- [Model and data loading](https://github.com/rifoagenadi/sycophancy/blob/main/utils.py): `utils.py`
- [Activation extraction](https://github.com/rifoagenadi/sycophancy/blob/main/probe/extract_activation.py): `probe/extract_activation.py`
- [Probe training](https://github.com/rifoagenadi/sycophancy/blob/main/probe/train.py): `probe/train.py`
- [Trained probes](https://github.com/rifoagenadi/sycophancy/blob/main/probe/train.py): `probe/train.py`
- [Projection statistics](https://github.com/rifoagenadi/sycophancy/blob/main/probe/compute_proj_std.py): `probe/compute_proj_std.py`
- [MHA intervention](https://github.com/rifoagenadi/sycophancy/blob/main/inference_mha.py): `inference_mha.py`
- [MLP intervention](https://github.com/rifoagenadi/sycophancy/blob/main/inference_mlp.py): `inference_mlp.py`
- [Residual intervention](https://github.com/rifoagenadi/sycophancy/blob/main/inference_residual.py): `inference_residual.py`

### Dialogue and steering

- [Disagreement dialogues](https://github.com/rifoagenadi/sycophancy/blob/main/uq/dialogue.py): `uq/dialogue.py`
- [Steering dialogues](https://github.com/rifoagenadi/sycophancy/blob/main/extension/dialogue.py): `extension/dialogue.py`
- [Dialogue activations](https://github.com/rifoagenadi/sycophancy/blob/main/extension/extract_activations.py): `extension/extract_activations.py`
- [Steering and ablation](https://github.com/rifoagenadi/sycophancy/blob/main/extension/steer.py): `extension/steer.py`

### Evaluation and analysis

- [Response judging](https://github.com/rifoagenadi/sycophancy/blob/main/extension/judge.py): `extension/judge.py`
- [Metrics](https://github.com/rifoagenadi/sycophancy/blob/main/extension/metrics.py): `extension/metrics.py`
- [Probe accuracy plots](https://github.com/rifoagenadi/sycophancy/blob/main/extension/diagram/probe_accuracy_plot.py): `extension/diagram/probe_accuracy_plot.py`
- [Disagreement plots](https://github.com/rifoagenadi/sycophancy/blob/main/extension/diagram/disagreement_type_plot.py): `extension/diagram/disagreement_type_plot.py`
- [Steering plots](https://github.com/rifoagenadi/sycophancy/blob/main/extension/diagram/steering_generalization_plot.py): `extension/diagram/steering_generalization_plot.py`

### Uncertainty quantification

- [UQ dialogue](https://github.com/rifoagenadi/sycophancy/blob/main/uq/dialogue.py): `uq/dialogue.py`
- [UQ judging](https://github.com/rifoagenadi/sycophancy/blob/main/uq/judge.py): `uq/judge.py`
- [UQ labeling](https://github.com/rifoagenadi/sycophancy/blob/main/uq/label.py): `uq/label.py`
- [UQ runner](https://github.com/rifoagenadi/sycophancy/blob/main/uq/run_uqlm.py): `uq/run_uqlm.py`
- [UQ visualization](https://github.com/rifoagenadi/sycophancy/blob/main/uq/visualize.py): `uq/visualize.py`

### Other components

- **Researcher**
- **TruthfulQA data**
- **Model provider**
- **Model responses**

## Connections

- Researcher → Probe training: trains probes (evidence: `README.md`)
- TruthfulQA data → Labeled examples: provides examples (evidence: `probe/probe_data_utils.py`)
- TruthfulQA data → Model and data loading: loads test data (evidence: `utils.py`)
- Model provider → Model and data loading: provides models (evidence: `utils.py`)
- Labeled examples → Probe training: labels data (evidence: `probe/train.py`)
- Model and data loading → Probe training: loads model (evidence: `probe/train.py`)
- Probe training → Activation extraction: extracts activations (evidence: `probe/train.py`)
- Probe training → Trained probes: saves weights (evidence: `README.md`)
- Trained probes → Projection statistics: projects directions (evidence: `probe/compute_proj_std.py`)
- Trained probes → MHA intervention: loads directions (evidence: `inference_mha.py`)
- Trained probes → MLP intervention: loads directions (evidence: `inference_mlp.py`)
- Trained probes → Residual intervention: loads directions (evidence: `inference_residual.py`)
- Model and data loading → MHA intervention: loads model and data (evidence: `inference_mha.py`)
- Model and data loading → MLP intervention: loads model and data (evidence: `inference_mlp.py`)
- Model and data loading → Residual intervention: loads model and data (evidence: `inference_residual.py`)
- MHA intervention → Model responses: writes predictions (evidence: `README.md`)
- MLP intervention → Model responses: writes predictions (evidence: `inference_mlp.py`)
- Residual intervention → Model responses: writes predictions (evidence: `inference_residual.py`)
- TruthfulQA data → Disagreement dialogues: loads questions (evidence: `uq/dialogue.py`)
- Disagreement dialogues → Model responses: writes dialogues (evidence: `uq/dialogue.py`)
- TruthfulQA data → Dialogue activations: builds dialogue samples (evidence: `extension/extract_activations.py`)
- Dialogue activations → Steering and ablation: supplies probe inputs (evidence: `extension/steer.py`)
- Disagreement dialogues → Steering and ablation: dialogue reference (evidence: `extension/steer.py`)
- Researcher → Response judging: evaluates responses
- Response judging → Metrics: provides judgments
- Metrics → Probe accuracy plots: plots accuracy
- Metrics → Disagreement plots: plots disagreement
- Metrics → Steering plots: plots generalization
- Researcher → UQ dialogue: runs UQ dialogue
- UQ dialogue → UQ judging: judgment workflow
- UQ judging → UQ labeling: labeling workflow
- UQ runner → UQ visualization: analysis workflow