# apolloresearch/deception-detection architecture

> Architecture diagram of the GitHub repository apolloresearch/deception-detection, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-03-28). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/apolloresearch/deception-detection
- Repository: https://github.com/apolloresearch/deception-detection

## Overview

##### Main Components
- Data Management:
- Data Storage: Represent the “data” directory that holds prompts, rollouts, trajectories, and training examples.
- Data Processing Modules: Highlight the “deception_detection/data” package where various dataset subclasses are implemented to process raw files.
- Core Experiment Modules:
- Experimental Orchestration & Configuration: Include the “deception_detection/experiment.py” as the central driver of experiments and its companion config files in “deception_detection/scripts/configs”.
- Experiment Entrypoints: Also note the experiment script at “deception_detection/scripts/experiment.py”.
- Detection & Probing Components:
- Detector Modules: Identify “deception_detection/detectors.py”, “deception_detection/models.py”, and “deception_detection/sae.py” as components for running linear probes and related detection mechanisms.
- Metrics and Prompt Scoring: Include “deception_detection/metrics.py” and “deception_detection/prompt_scorer.py” that evaluate system performance.
- Results & Visualization:
- Results Processing: Note the “deception_detection/results” modules for generating visualizations, plots, and HTML outputs.
- Notebooks and Analysis: Use the “deception_detection/notebooks” folder that contains exploratory analysis and debugging scripts.
- Utility & Integration:
- Infrastructure & Logging: The “deception_detection/log.py”, “deception_detection/utils.py”, and files like “pyproject.toml”, “Makefile”, and CI/CD configurations in “.github” reflect standard development utilities.
- External API Integration: The README and .env recommendations show dependencies on external APIs (e.g., OpenAI, Hugging Face, and Goodfire).

##### Relationships and Interactions
- Data flows from the raw files in the “data” directory into the dataset loaders implemented in “deception_detection/data”.
- The experiment module orchestrates the processing and evaluation, invoking the detector and probe components to run experiments based on the datasets.
- Detectors use computed metrics, logs, and prompts (from both configurations and generated rollouts) to assess model deception.
- Experiment results pass output into the results and visualization modules, which then produce plots, tables, and reports saved under folders like “example_results” and “deception_detection/results”.
- Supporting scripts under “deception_detection/scripts” act as the glue to perform tasks such as generating rollouts, converting facts to Q&A, or running analysis pipelines.

##### Architectural Patterns and Design Principles
- Modularity: The clear separation of concerns is evident—data ingestion, experiment orchestration, detection logic, and result visualization are each in separate modules/packages.
- Configurability: The use of YAML config files (in “deception_detection/scripts/configs”) and environment variables for API keys implies an emphasis on configurable experiment parameters.
- Extensibility: The presence of multiple dataset-specific files and generation scripts (for roleplaying, insider trading, etc.) shows that the codebase is designed to plug in new conditions or experiment types.
- Reusability: Utility modules (logging, metrics, and shared functions) help in avoiding code duplication throughout the experimental workflows.

## Diagram (Mermaid)

```mermaid
graph TD
    subgraph "Deception Detection System"
        %% Data Layer
        subgraph "Data Layer"
            DS["Data Storage"]:::data
            DP["Data Processing Modules"]:::data
        end

        %% Experiment Orchestration Layer
        subgraph "Experiment Orchestration"
            EO["Experimental Orchestration & Configuration"]:::experiment
            EE["Experiment Entrypoints"]:::experiment
        end

        %% Detection & Metrics Layer
        subgraph "Detection & Metrics"
            DM["Detector Modules"]:::detection
            MS["Metrics & Prompt Scoring"]:::detection
        end

        %% Results & Visualization Layer
        subgraph "Results & Visualization"
            RP["Results Processing"]:::results
            NA["Notebooks & Analysis"]:::results
        end

        %% Integration & Utilities Layer
        subgraph "Integration & Utilities"
            IL["Infrastructure & Logging"]:::utility
            EA["External API Integration"]:::external
        end
    end

    %% Data flow
    DS -->|"raw_data"| DP
    DP -->|"load_datasets"| EO
    DP -->|"prepare_data"| EE

    %% Experiment triggers Detection & Metrics
    EO -->|"run_experiments"| DM
    EO -->|"evaluate_experiments"| MS

    %% Detection outputs feed Results
    DM -->|"detection_output"| RP
    MS -->|"performance_metrics"| RP
    RP -->|"analysis_plots"| NA

    %% Logging from various modules
    EO -->|"logs"| IL
    DM -->|"logs"| IL
    RP -->|"logs"| IL

    %% External API integration with Detector Modules
    DM -->|"API_calls"| EA

    %% Click Events
    click DS "https://github.com/apolloresearch/deception-detection/tree/main//data"
    click DP "https://github.com/apolloresearch/deception-detection/tree/main//deception_detection/data"
    click EO "https://github.com/apolloresearch/deception-detection/blob/main//deception_detection/experiment.py"
    click EO "https://github.com/apolloresearch/deception-detection/tree/main//deception_detection/scripts/configs"
    click EE "https://github.com/apolloresearch/deception-detection/blob/main//deception_detection/scripts/experiment.py"
    click DM "https://github.com/apolloresearch/deception-detection/blob/main//deception_detection/detectors.py"
    click DM "https://github.com/apolloresearch/deception-detection/blob/main//deception_detection/models.py"
    click DM "https://github.com/apolloresearch/deception-detection/blob/main//deception_detection/sae.py"
    click MS "https://github.com/apolloresearch/deception-detection/blob/main//deception_detection/metrics.py"
    click MS "https://github.com/apolloresearch/deception-detection/blob/main//deception_detection/prompt_scorer.py"
    click RP "https://github.com/apolloresearch/deception-detection/tree/main//deception_detection/results"
    click NA "https://github.com/apolloresearch/deception-detection/tree/main//deception_detection/notebooks"
    click IL "https://github.com/apolloresearch/deception-detection/blob/main//deception_detection/log.py"
    click IL "https://github.com/apolloresearch/deception-detection/blob/main//deception_detection/utils.py"
    click IL "https://github.com/apolloresearch/deception-detection/tree/main//Makefile"
    click IL "https://github.com/apolloresearch/deception-detection/blob/main//pyproject.toml"
    click IL "https://github.com/apolloresearch/deception-detection/blob/main//.github"
    click EA "https://github.com/apolloresearch/deception-detection/blob/main//deception_detection/goodfire_client.py"

    %% Styles
    classDef data fill:#AED6F1,stroke:#1B4F72,stroke-width:2px;
    classDef experiment fill:#F9E79F,stroke:#B7950B,stroke-width:2px;
    classDef detection fill:#ABEBC6,stroke:#1D8348,stroke-width:2px;
    classDef results fill:#F5B7B1,stroke:#922B21,stroke-width:2px;
    classDef utility fill:#E8DAEF,stroke:#6C3483,stroke-width:2px;
    classDef external fill:#FAD7A0,stroke:#D35400,stroke-width:2px;
```
