# eleutherai/delphi architecture

> Architecture diagram of the GitHub repository eleutherai/delphi, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-08-01). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/eleutherai/delphi
- Repository: https://github.com/eleutherai/delphi

## Overview

##### Main Architectural Layers
a. **Entry Points (CLI & Programmatic API)**
- `delphi/__main__.py` (CLI invocation)
- Any user code that imports `delphi.pipeline`, `delphi.latents`, etc.
b. **Orchestration & Pipeline**
- `delphi/pipeline.py` (Pipeline class, process_wrapper, async runner)
c. **Core Modules**
- **Clients** (`delphi/clients`) – adapters to LLM backends:
- `Offline` (local VLLM)
- `OpenRouter` (API)
- **Latent Handling** (`delphi/latents`) – caching, loading, constructing contexts:
- `LatentCache`, `LatentDataset`, `ConstructorConfig`, `SamplerConfig`
- **Explainers** (`delphi/explainers`) – build prompts + call model:
- DefaultExplainer, ContrastiveExplainer, NoOp, SingleToken, etc.
- **Scorers** (`delphi/scorers`) – evaluate explanations:
- ClassifierScorers (detection, fuzzing, intruder)
- EmbeddingScorer
- SimulatorScorer (OpenAI auto‐interp fork)
- SurprisalScorer
- **Sparse Coders** (`delphi/sparse_coders`) – AE model wrappers and loading
- **Configuration & Utilities**
- `delphi/config.py`
- `delphi/utils.py`, logging (`delphi/log`)
d. **Data Stores & File Artifacts**
- Activation caches (`.safetensors` shards)
- Embedding caches (FAISS index)
- Generated explanations & scores (JSON/text files)
- Logs, histograms, confusion matrices
e. **External Dependencies & Services**
- HuggingFace Datasets & Models
- FAISS (semantic search)
- OpenRouter / OpenAI API
- VLLM local model server

##### Component Relationships & Data Flows
- **Data Flow**:
1. **Dataset Load** → tokens
2. **LatentCache** (model + tokens) → activation shards on disk
3. **LatentDataset** (constructor + sampler) → context windows
4. **Pipeline** orchestrates steps:
a. **ExplainerPipe**: contexts → Explainer → explanations
b. **ScorerPipe**: (contexts + explanations) → Scorers → scores
5. **ResultAnalysis**: collate metrics, generate plots
- **Control Flow**: CLI or user script triggers Pipeline → async tasks
- **Dependencies**:
- Explainers & Scorers depend on a **Client**
- LatentDataset depends on caches + config
- FAISS index built by Constructor if enabled

##### Architectural Patterns & Principles
- **Modular Pipeline** – each stage is a pluggable “pipe”
- **Adapter Pattern** for LLM clients (Offline vs. OpenRouter)
- **Separation of Concerns**: caching vs. constructing vs. explaining vs. scoring
- **Asynchronous Processing** with `asyncio` for parallel API/model calls
- **Configuration-driven** (dataclasses in `config.py`)
- **Extensibility Points**: add new Explainer, Scorer, or Client by subclassing

## Diagram (Mermaid)

```mermaid
graph TD
    %% Entry
    CLI["CLI / Programmatic API"]:::entry
    CLI -->|control| PIPE["Pipeline Orchestrator\n(delphi/pipeline.py)"]:::core
    click CLI "https://github.com/eleutherai/delphi/blob/main/delphi/__main__.py"
    click PIPE "https://github.com/eleutherai/delphi/blob/main/delphi/pipeline.py"

    %% Data Source
    DS["Dataset\n(HuggingFace)"]:::external
    DS -->|load tokens| LD

    %% Orchestration to Latents
    PIPE -->|1 tokens| LD["LatentDataset\n(delphi/latents/latents.py,\n delphi/latents/samplers.py)"]:::core
    click LD "https://github.com/eleutherai/delphi/blob/main/delphi/latents/latents.py"
    PIPE -->|collect activations| LC["LatentCache\n(delphi/latents/cache.py,\n delphi/latents/collect_activations.py)"]:::core
    click LC "https://github.com/eleutherai/delphi/blob/main/delphi/latents/cache.py"

    LC -->|writes| AS["Activation Shards\n(.safetensors)"]:::datastore

    %% Explainer Pipeline
    LD -->|construct contexts| EXPIPE["ExplainerPipe"]:::core
    subgraph "ExplainerPipe"
      EXPL1["DefaultExplainer\ndelphi/explainers/default/default.py"]:::core
      click EXPL1 "https://github.com/eleutherai/delphi/blob/main/delphi/explainers/default/default.py"
      EXPL2["PromptBuilder\n.../prompt_builder.py"]:::core
      click EXPL2 "https://github.com/eleutherai/delphi/blob/main/delphi/explainers/default/prompt_builder.py"
      EXPL3["ContrastiveExplainer\ndelphi/explainers/contrastive_explainer.py"]:::core
      click EXPL3 "https://github.com/eleutherai/delphi/blob/main/delphi/explainers/contrastive_explainer.py"
      EXPL4["NoOpExplainer\ndelphi/explainers/no_op_explainer.py"]:::core
      click EXPL4 "https://github.com/eleutherai/delphi/blob/main/delphi/explainers/no_op_explainer.py"
      EXPL5["SingleTokenExplainer\n.../single_token_explainer.py"]:::core
      click EXPL5 "https://github.com/eleutherai/delphi/blob/main/delphi/explainers/single_token_explainer.py"
    end
    EXPIPE -->|explanations| SCPIPE

    %% Scorer Pipeline
    subgraph "ScorerPipe"
      SBASE["ScorerBase\ndelphi/scorers/scorer.py"]:::core
      click SBASE "https://github.com/eleutherai/delphi/blob/main/delphi/scorers/scorer.py"
      SCLASS["ClassifierScorers\n.../classifier.py"]:::core
      click SCLASS "https://github.com/eleutherai/delphi/blob/main/delphi/scorers/classifier/classifier.py"
      SDET["Detection\n.../detection.py"]:::core
      click SDET "https://github.com/eleutherai/delphi/blob/main/delphi/scorers/classifier/detection.py"
      SFUZZ["Fuzzing\n.../fuzz.py"]:::core
      click SFUZZ "https://github.com/eleutherai/delphi/blob/main/delphi/scorers/classifier/fuzz.py"
      SINTR["Intruder\n.../intruder.py"]:::core
      click SINTR "https://github.com/eleutherai/delphi/blob/main/delphi/scorers/classifier/intruder.py"
      SEMP["EmbeddingScorer\n.../embedding.py"]:::core
      click SEMP "https://github.com/eleutherai/delphi/blob/main/delphi/scorers/embedding/embedding.py"
      SSIM["SimulatorScorer\noai_autointerp"]:::core
      click SSIM "https://github.com/eleutherai/delphi/blob/main/delphi/scorers/simulator/oai_simulator.py"
      SSURP["SurprisalScorer\nsurprisal.py"]:::core
      click SSURP "https://github.com/eleutherai/delphi/blob/main/delphi/scorers/surprisal/surprisal.py"
    end
    SCPIPE -->|scores| RA["ResultAnalysis & Logging\n(delphi/log/result_analysis.py)"]:::core
    click RA "https://github.com/eleutherai/delphi/blob/main/delphi/log/result_analysis.py"

    %% Clients (Adapters)
    subgraph "Clients (Adapters)"
      CBASE["ClientBase\ndelphi/clients/client.py"]:::core
      click CBASE "https://github.com/eleutherai/delphi/blob/main/delphi/clients/client.py"
      COFF["OfflineAdapter\ndelphi/clients/offline.py"]:::core
      click COFF "https://github.com/eleutherai/delphi/blob/main/delphi/clients/offline.py"
      COPE["OpenRouterAdapter\n.../openrouter.py"]:::core
      click COPE "https://github.com/eleutherai/delphi/blob/main/delphi/clients/openrouter.py"
    end
    EXPIPE -->|uses| CBASE
    SCPIPE -->|uses| CBASE

    %% External Services & Data Stores
    FAISS["FAISS Index\n(embedding cache)"]:::datastore
    VLLM["VLLM Server"]:::external
    OA["OpenAI / OpenRouter API"]:::external

    CBASE -->|calls| OA
    COFF -->|connects| VLLM
    LD -->|build index| FAISS

    %% Classes
    classDef core fill:#D6EAF8,stroke:#1B4F72;
    classDef external fill:#D4EFDF,stroke:#196F3D;
    classDef datastore fill:#FCF3CF,stroke:#B9770E;
    classDef entry fill:#E8DAEF,stroke:#6C3483;
```
