# ukgovernmentbeis/control-arena architecture

> Architecture diagram of the GitHub repository ukgovernmentbeis/control-arena, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-03-31). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/ukgovernmentbeis/control-arena
- Repository: https://github.com/ukgovernmentbeis/control-arena

## Overview

Based on our review of both the file tree and the README, ControlArena is a research prototype framework aimed at mimicking real-world, high-stakes deployment environments and supporting control evaluations across multiple complex tasks. It incorporates a modular design with several major sectors such as protocols, settings (for different environments like Kubernetes-based “Infra”, as well as “Bash” and “Apps”), CLI tools, task definitions, and supporting libraries for logging, auditing, and utility functions.

##### Main Components
- Overall “ControlArena” core, which includes:
- The protocols layer (in control_arena/protocols) that defines how agents communicate, monitor activities, and enforce policies.
- The settings layer (in control_arena/settings) that provides definitions for different evaluation environments. Notably:
⁃ “Infra” for Kubernetes infrastructure (with a rich hierarchy of main and side tasks),
⁃ “Bash” for executing bash commands,
⁃ “Apps” for APPS-based coding tasks.
- The CLI and tooling components. For example, control_arena/cli hosts the main CLI functionality and there is a dedicated “human-trajectories” CLI in the scripts/human_trajectories directory.
- Log management modules (control_arena/log_store) which likely provide mechanisms for logging to GitHub, servers, etc.
- Utility helpers and testing frameworks (including tests directories, conftest.py, and utility scripts).
- Devcontainer and other configuration files that support environment setup, dependency management (like uv and Dockerfiles) and CI/CD (GitHub workflows).

##### Relationships and Interactions
- The CLI interacts with various components of the framework:
- It triggers environment setup, evaluation runs (e.g. via “uv run inspect eval …”), and launches the different settings.
- It also connects with protocol modules to choose the evaluation protocol (e.g. simple protocol, tm protocol, etc.) and with policies (attack, honest, fixed).
- Protocol modules and task definitions should be drawn as separate submodules, illustrating that protocols are built using smaller parts (like monitors and policies) that reside in protocol_parts.
- Bidirectional data flows:
- For example, control loop and task execution feed data into logging modules which record evaluations.
- The human trajectories CLI tools extract and generate evaluation trajectories that are later consumed by the evaluation engine.

##### Important Architectural Patterns
- Modularity: The design separates domain-specific logic into clearly isolated directories (protocols vs settings vs CLI vs tooling).
- Layering: There is a separation between core logic (control_arena) and environment/task-specific configuration (control_arena/settings), and a clear division between runtime interaction (CLI and human trajectories) and infrastructure tasks.
- Extensibility: The protocols module is built to be extended via subcomponents (as seen in the protocol_parts folder) so drawing these as “pluggable” modules is useful.

##### Relevant Technologies and Frameworks
- Python as the primary language, with dependency management by “uv”.
- Infrastructure tools: Docker (incorporated via Dockerfiles, devcontainer configuration, and infra/docker folder), Kubernetes (kubectl, kind, helm), and CI/CD pipelines (GitHub workflows).
- Testing and code quality via pre-commit configurations, pytest tests (tests directories), and linting configurations (e.g. .markdownlint.jsonc).

##### Capturing Essential Architectural Detail
- Separate and detail as many components as possible instead of lumping functionalities together.
- A block representing “control_arena” (with sub-blocks for protocols, settings, cli, log_store).
- A block for “scripts” and “tests”.
- Configuration blocks for “.devcontainer”, Dockerfiles, CI/CD files.
- Interactions: for example, the CLI triggers evaluations that use protocols defined in control_arena/protocols, which then interact with settings in control_arena/settings/infra, and results are logged in control_arena/log_store.

This breakdown should help ensure all essential architectural elements are captured in an easily understandable visual format.

## Diagram (Mermaid)

```mermaid
graph TD
  %% ControlArena Core Block
  subgraph "ControlArena Core"
    CA["ControlArena Core"]:::core
    CT["CLI Engine"]:::cli
    PC["Protocols Layer"]:::protocol
    PP["Protocol Parts (Monitors & Policies)"]:::protocolparts
    subgraph "Settings Layer"
      SI["Infra Environment"]:::infra
      SB["Bash Environment"]:::bash
      SA["Apps Environment"]:::apps
    end
    LS["Log Store"]:::log
    UT["Utility Helpers"]:::utility
  end

  %% External and Supporting Components
  HTC["Human Trajectories CLI"]:::cli2
  TS["Testing & Utility Scripts"]:::test
  DS["Devcontainer & Docker Setup"]:::config
  CI["CI/CD Pipelines"]:::config2
  EX["External Infra Tools (Kubernetes,Helm,Kind)"]:::external

  %% Internal Core Connections
  CA --> CT
  CA --> PC
  CA --> LS
  CA --> UT
  PC --> PP

  %% CLI triggering actions to protocols and settings
  CT -->|"triggers"| PC
  CT -->|"triggers"| SI
  CT -->|"triggers"| SB
  CT -->|"triggers"| SA

  %% Protocol layer logs evaluations
  PC -->|"logs_to"| LS

  %% Settings Infra calls external tools
  SI -->|"calls"| EX

  %% Human Trajectories integration with CLI
  HTC -->|"integrates_with"| CT

  %% Testing supports utilities
  TS -->|"supports"| UT

  %% Devcontainer setup configures CLI operations
  DS -->|"setup"| CT

  %% CI/CD pipelines deploy via Devcontainer setup
  CI -->|"deploys_to"| DS

  %% Utility helpers assist CLI operations
  UT -->|"assists"| CT

  %% Click Events
  click CA "https://github.com/ukgovernmentbeis/control-arena/tree/main/control_arena/"
  click PC "https://github.com/ukgovernmentbeis/control-arena/tree/main/control_arena/protocols/"
  click PP "https://github.com/ukgovernmentbeis/control-arena/tree/main/control_arena/protocols/protocol_parts/"
  click SI "https://github.com/ukgovernmentbeis/control-arena/tree/main/control_arena/settings/infra/"
  click SB "https://github.com/ukgovernmentbeis/control-arena/tree/main/control_arena/settings/bash/"
  click SA "https://github.com/ukgovernmentbeis/control-arena/tree/main/control_arena/settings/apps/"
  click CT "https://github.com/ukgovernmentbeis/control-arena/tree/main/control_arena/cli/"
  click HTC "https://github.com/ukgovernmentbeis/control-arena/tree/main/scripts/human_trajectories/"
  click LS "https://github.com/ukgovernmentbeis/control-arena/tree/main/control_arena/log_store/"
  click UT "https://github.com/ukgovernmentbeis/control-arena/tree/main/control_arena/tools/"
  click TS "https://github.com/ukgovernmentbeis/control-arena/tree/main/tests/"
  click DS "https://github.com/ukgovernmentbeis/control-arena/tree/main/.devcontainer/"
  click CI "https://github.com/ukgovernmentbeis/control-arena/tree/main/.github/workflows/"

  %% Class Definitions with Colors
  classDef core fill:#f9e79f,stroke:#7d6608,stroke-width:2px;
  classDef protocol fill:#aed6f1,stroke:#1b4f72,stroke-width:2px;
  classDef protocolparts fill:#85c1e9,stroke:#154360,stroke-width:2px;
  classDef infra fill:#a9dfbf,stroke:#196f3d,stroke-width:2px;
  classDef bash fill:#d6eaf8,stroke:#1e87d8,stroke-width:2px;
  classDef apps fill:#abebc6,stroke:#117a65,stroke-width:2px;
  classDef cli fill:#f5b7b1,stroke:#943126,stroke-width:2px;
  classDef cli2 fill:#f5b7b1,stroke:#943126,stroke-width:2px;
  classDef log fill:#fcf3cf,stroke:#7d6608,stroke-width:2px;
  classDef utility fill:#d7bde2,stroke:#6c3483,stroke-width:2px;
  classDef test fill:#fad7a0,stroke:#b9770e,stroke-width:2px;
  classDef config fill:#d5f5e3,stroke:#117864,stroke-width:2px;
  classDef config2 fill:#fcf3cf,stroke:#b7950b,stroke-width:2px;
  classDef external fill:#e8daef,stroke:#6c3483,stroke-width:2px;
```
