# viswatejam702-tech/inspect_ai architecture

> Architecture diagram of the GitHub repository viswatejam702-tech/inspect_ai, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-11-15). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/viswatejam702-tech/inspect_ai
- Repository: https://github.com/viswatejam702-tech/inspect_ai

## Overview

Inspect is a Python framework for running LLM evaluations, with built-in prompt, tool, multi-turn, and model-graded evaluation capabilities, plus extension points. The principal workflow is evaluation launch, task/sample execution, model and tool interaction, scoring, and result inspection. The tree also shows ACP agent connectivity, sandbox tools, and evaluation-log analysis. Several workflow links are necessarily unverified because their implementation was not sampled; the graph uses dashed, uncited edges for those relationships.

Main entry points: [`src/inspect_ai/_eval/run.py`](https://github.com/viswatejam702-tech/inspect_ai/blob/main/src/inspect_ai/_eval/run.py) (Evaluation runner), [`src/inspect_ai/_eval/task/run.py`](https://github.com/viswatejam702-tech/inspect_ai/blob/main/src/inspect_ai/_eval/task/run.py) (Task execution), [`src/inspect_sandbox_tools/src/inspect_sandbox_tools/_cli/server.py`](https://github.com/viswatejam702-tech/inspect_ai/blob/main/src/inspect_sandbox_tools/src/inspect_sandbox_tools/_cli/server.py) (Sandbox server), [`src/inspect_ai/agent/_acp/server.py`](https://github.com/viswatejam702-tech/inspect_ai/blob/main/src/inspect_ai/agent/_acp/server.py) (ACP server)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_run["Evaluation runtime"]
  node_eval_cli["Eval command<br/>[eval.py]"]
  node_eval_run["Evaluation runner<br/>[run.py]"]
  node_task_run["Task execution<br/>[run.py]"]
  node_dataset["Datasets<br/>[_dataset.py]"]
  node_scoring["Scoring<br/>[score.py]"]
end

subgraph group_interact["Models and agents"]
  node_model["Model interface<br/>[_model.py]"]
  node_chat["Chat messages<br/>[_chat_message.py]"]
  node_agents["Agent framework<br/>[_run.py]"]
  node_approval["Human approval<br/>[manager.py]"]
  node_acp_server["ACP server<br/>[server.py]"]
  node_acp_connection["ACP connection<br/>[connection.py]"]
  node_acp_transport["ACP transport<br/>[transport.py]"]
  node_acp_forward["Session forwarder<br/>[session_router.py]"]
  node_content_util["Content utilities<br/>[content.py]"]
  node_acp_guards["ACP guards<br/>[_guards.py]"]
end

subgraph group_state["Logs and control"]
  node_event["Evaluation events<br/>[_model.py]"]
  node_sample_state["Live sample state<br/>[_samples.py]"]
  node_logs[("Evaluation logs<br/>[database.py]")]
  node_control["Control server<br/>[server.py]"]
  node_analysis["Log analysis<br/>[extract.py]"]
  node_json_util["JSON utilities<br/>[json.py]"]
  node_discovery["Discovery utilities<br/>[discovery.py]"]
  node_sockets["Socket utilities<br/>[sockets.py]"]
end

subgraph group_interfaces["User interfaces"]
  node_cli["CLI dispatch<br/>[main.py]"]
  node_viewer["Web viewer<br/>[view.py]"]
  node_tui["Textual display<br/>[app.py]"]
  node_rich_util["Rich formatting<br/>[rich.py]"]
end

subgraph group_sandbox["Sandbox tools"]
  node_sandbox_cli["Sandbox CLI<br/>[main.py]"]
  node_sandbox_server["Sandbox server<br/>[server.py]"]
  node_sandbox_proxy["Agent bridge<br/>[proxy.py]"]
  node_sandbox_bash["Remote bash<br/>[_controller.py]"]
  node_sandbox_exec["Remote execution<br/>[_controller.py]"]
  node_sandbox_mcp["Remote MCP"]
end

node_user(("Evaluation author"))

node_user -.->|"launches"| node_cli
node_cli -->|"dispatches"| node_eval_cli
node_eval_cli -.->|"starts"| node_eval_run
node_eval_run -.->|"executes"| node_task_run
node_task_run -.->|"loads samples"| node_dataset
node_task_run -.->|"requests completion"| node_model
node_task_run -.->|"runs agent"| node_agents
node_task_run -.->|"scores results"| node_scoring
node_model -.->|"uses messages"| node_chat
node_agents -.->|"requests review"| node_approval
node_acp_server -->|"handles connections"| node_acp_connection
node_acp_connection -->|"uses guards"| node_acp_guards
node_acp_server -->|"uses discovery"| node_discovery
node_acp_server -->|"uses sockets"| node_sockets
node_acp_forward -->|"forwards updates"| node_acp_transport
node_eval_run -.->|"reports live state"| node_sample_state
node_eval_run -.->|"records results"| node_logs
node_eval_run -.->|"hosts control"| node_control
node_logs -.->|"feeds analysis"| node_analysis
node_logs -.->|"shown in viewer"| node_viewer
node_cli -.->|"offers display"| node_tui
node_sandbox_cli -->|"dispatches proxy"| node_sandbox_proxy
node_sandbox_cli -->|"starts server"| node_sandbox_server
node_sandbox_server -->|"registers methods"| node_sandbox_bash
node_sandbox_server -->|"registers methods"| node_sandbox_exec
node_sandbox_server -->|"registers methods"| node_sandbox_mcp

click node_cli "https://github.com/viswatejam702-tech/inspect_ai/blob/main/src/inspect_ai/_cli/main.py"
click node_eval_cli "https://github.com/viswatejam702-tech/inspect_ai/blob/main/src/inspect_ai/_cli/eval.py"
click node_eval_run "https://github.com/viswatejam702-tech/inspect_ai/blob/main/src/inspect_ai/_eval/run.py"
click node_task_run "https://github.com/viswatejam702-tech/inspect_ai/blob/main/src/inspect_ai/_eval/task/run.py"
click node_dataset "https://github.com/viswatejam702-tech/inspect_ai/blob/main/src/inspect_ai/dataset/_dataset.py"
click node_scoring "https://github.com/viswatejam702-tech/inspect_ai/blob/main/src/inspect_ai/_eval/score.py"
click node_model "https://github.com/viswatejam702-tech/inspect_ai/blob/main/src/inspect_ai/model/_model.py"
click node_chat "https://github.com/viswatejam702-tech/inspect_ai/blob/main/src/inspect_ai/model/_chat_message.py"
click node_agents "https://github.com/viswatejam702-tech/inspect_ai/blob/main/src/inspect_ai/agent/_run.py"
click node_approval "https://github.com/viswatejam702-tech/inspect_ai/blob/main/src/inspect_ai/approval/_human/manager.py"
click node_acp_server "https://github.com/viswatejam702-tech/inspect_ai/blob/main/src/inspect_ai/agent/_acp/server.py"
click node_acp_connection "https://github.com/viswatejam702-tech/inspect_ai/blob/main/src/inspect_ai/agent/_acp/connection.py"
click node_acp_transport "https://github.com/viswatejam702-tech/inspect_ai/blob/main/src/inspect_ai/agent/_acp/transport.py"
click node_acp_forward "https://github.com/viswatejam702-tech/inspect_ai/blob/main/src/inspect_ai/agent/_acp/session_router.py"
click node_event "https://github.com/viswatejam702-tech/inspect_ai/blob/main/src/inspect_ai/event/_model.py"
click node_sample_state "https://github.com/viswatejam702-tech/inspect_ai/blob/main/src/inspect_ai/log/_samples.py"
click node_logs "https://github.com/viswatejam702-tech/inspect_ai/blob/main/src/inspect_ai/log/_recorders/buffer/database.py"
click node_control "https://github.com/viswatejam702-tech/inspect_ai/blob/main/src/inspect_ai/_control/server.py"
click node_analysis "https://github.com/viswatejam702-tech/inspect_ai/blob/main/src/inspect_ai/analysis/_dataframe/extract.py"
click node_viewer "https://github.com/viswatejam702-tech/inspect_ai/blob/main/src/inspect_ai/_view/view.py"
click node_tui "https://github.com/viswatejam702-tech/inspect_ai/blob/main/src/inspect_ai/_display/textual/app.py"
click node_sandbox_cli "https://github.com/viswatejam702-tech/inspect_ai/blob/main/src/inspect_sandbox_tools/src/inspect_sandbox_tools/_cli/main.py"
click node_sandbox_server "https://github.com/viswatejam702-tech/inspect_ai/blob/main/src/inspect_sandbox_tools/src/inspect_sandbox_tools/_cli/server.py"
click node_sandbox_proxy "https://github.com/viswatejam702-tech/inspect_ai/blob/main/src/inspect_sandbox_tools/src/inspect_sandbox_tools/_agent_bridge/proxy.py"
click node_sandbox_bash "https://github.com/viswatejam702-tech/inspect_ai/blob/main/src/inspect_sandbox_tools/src/inspect_sandbox_tools/_remote_tools/_bash_session/_controller.py"
click node_sandbox_exec "https://github.com/viswatejam702-tech/inspect_ai/blob/main/src/inspect_sandbox_tools/src/inspect_sandbox_tools/_remote_tools/_exec_remote/_controller.py"
click node_sandbox_mcp "https://github.com/viswatejam702-tech/inspect_ai/blob/main/src/inspect_sandbox_tools/src/inspect_sandbox_tools/_remote_tools/_mcp/mcp_server_session.py"
click node_json_util "https://github.com/viswatejam702-tech/inspect_ai/blob/main/src/inspect_ai/_util/json.py"
click node_content_util "https://github.com/viswatejam702-tech/inspect_ai/blob/main/src/inspect_ai/_util/content.py"
click node_discovery "https://github.com/viswatejam702-tech/inspect_ai/blob/main/src/inspect_ai/_util/discovery.py"
click node_sockets "https://github.com/viswatejam702-tech/inspect_ai/blob/main/src/inspect_ai/_util/sockets.py"
click node_rich_util "https://github.com/viswatejam702-tech/inspect_ai/blob/main/src/inspect_ai/_util/rich.py"
click node_acp_guards "https://github.com/viswatejam702-tech/inspect_ai/blob/main/src/inspect_ai/agent/_acp/_guards.py"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_eval_cli,node_eval_run,node_task_run,node_dataset,node_scoring toneBlue
class node_model,node_chat,node_agents,node_approval,node_acp_server,node_acp_connection,node_acp_transport,node_acp_forward,node_content_util,node_acp_guards toneAmber
class node_event,node_sample_state,node_logs,node_control,node_analysis,node_json_util,node_discovery,node_sockets toneMint
class node_cli,node_viewer,node_tui,node_rich_util toneRose
class node_sandbox_cli,node_sandbox_server,node_sandbox_proxy,node_sandbox_bash,node_sandbox_exec,node_sandbox_mcp,node_user toneIndigo
```

## Components

### Evaluation runtime

- [Eval command](https://github.com/viswatejam702-tech/inspect_ai/blob/main/src/inspect_ai/_cli/eval.py): `src/inspect_ai/_cli/eval.py`
- [Evaluation runner](https://github.com/viswatejam702-tech/inspect_ai/blob/main/src/inspect_ai/_eval/run.py): `src/inspect_ai/_eval/run.py`
- [Task execution](https://github.com/viswatejam702-tech/inspect_ai/blob/main/src/inspect_ai/_eval/task/run.py): `src/inspect_ai/_eval/task/run.py`
- [Datasets](https://github.com/viswatejam702-tech/inspect_ai/blob/main/src/inspect_ai/dataset/_dataset.py): `src/inspect_ai/dataset/_dataset.py`
- [Scoring](https://github.com/viswatejam702-tech/inspect_ai/blob/main/src/inspect_ai/_eval/score.py): `src/inspect_ai/_eval/score.py`

### Models and agents

- [Model interface](https://github.com/viswatejam702-tech/inspect_ai/blob/main/src/inspect_ai/model/_model.py): `src/inspect_ai/model/_model.py`
- [Chat messages](https://github.com/viswatejam702-tech/inspect_ai/blob/main/src/inspect_ai/model/_chat_message.py): `src/inspect_ai/model/_chat_message.py`
- [Agent framework](https://github.com/viswatejam702-tech/inspect_ai/blob/main/src/inspect_ai/agent/_run.py): `src/inspect_ai/agent/_run.py`
- [Human approval](https://github.com/viswatejam702-tech/inspect_ai/blob/main/src/inspect_ai/approval/_human/manager.py): `src/inspect_ai/approval/_human/manager.py`
- [ACP server](https://github.com/viswatejam702-tech/inspect_ai/blob/main/src/inspect_ai/agent/_acp/server.py): `src/inspect_ai/agent/_acp/server.py`
- [ACP connection](https://github.com/viswatejam702-tech/inspect_ai/blob/main/src/inspect_ai/agent/_acp/connection.py): `src/inspect_ai/agent/_acp/connection.py`
- [ACP transport](https://github.com/viswatejam702-tech/inspect_ai/blob/main/src/inspect_ai/agent/_acp/transport.py): `src/inspect_ai/agent/_acp/transport.py`
- [Session forwarder](https://github.com/viswatejam702-tech/inspect_ai/blob/main/src/inspect_ai/agent/_acp/session_router.py): `src/inspect_ai/agent/_acp/session_router.py`
- [Content utilities](https://github.com/viswatejam702-tech/inspect_ai/blob/main/src/inspect_ai/_util/content.py): `src/inspect_ai/_util/content.py`
- [ACP guards](https://github.com/viswatejam702-tech/inspect_ai/blob/main/src/inspect_ai/agent/_acp/_guards.py): `src/inspect_ai/agent/_acp/_guards.py`

### Logs and control

- [Evaluation events](https://github.com/viswatejam702-tech/inspect_ai/blob/main/src/inspect_ai/event/_model.py): `src/inspect_ai/event/_model.py`
- [Live sample state](https://github.com/viswatejam702-tech/inspect_ai/blob/main/src/inspect_ai/log/_samples.py): `src/inspect_ai/log/_samples.py`
- [Evaluation logs](https://github.com/viswatejam702-tech/inspect_ai/blob/main/src/inspect_ai/log/_recorders/buffer/database.py): `src/inspect_ai/log/_recorders/buffer/database.py`
- [Control server](https://github.com/viswatejam702-tech/inspect_ai/blob/main/src/inspect_ai/_control/server.py): `src/inspect_ai/_control/server.py`
- [Log analysis](https://github.com/viswatejam702-tech/inspect_ai/blob/main/src/inspect_ai/analysis/_dataframe/extract.py): `src/inspect_ai/analysis/_dataframe/extract.py`
- [JSON utilities](https://github.com/viswatejam702-tech/inspect_ai/blob/main/src/inspect_ai/_util/json.py): `src/inspect_ai/_util/json.py`
- [Discovery utilities](https://github.com/viswatejam702-tech/inspect_ai/blob/main/src/inspect_ai/_util/discovery.py): `src/inspect_ai/_util/discovery.py`
- [Socket utilities](https://github.com/viswatejam702-tech/inspect_ai/blob/main/src/inspect_ai/_util/sockets.py): `src/inspect_ai/_util/sockets.py`

### User interfaces

- [CLI dispatch](https://github.com/viswatejam702-tech/inspect_ai/blob/main/src/inspect_ai/_cli/main.py): `src/inspect_ai/_cli/main.py`
- [Web viewer](https://github.com/viswatejam702-tech/inspect_ai/blob/main/src/inspect_ai/_view/view.py): `src/inspect_ai/_view/view.py`
- [Textual display](https://github.com/viswatejam702-tech/inspect_ai/blob/main/src/inspect_ai/_display/textual/app.py): `src/inspect_ai/_display/textual/app.py`
- [Rich formatting](https://github.com/viswatejam702-tech/inspect_ai/blob/main/src/inspect_ai/_util/rich.py): `src/inspect_ai/_util/rich.py`

### Sandbox tools

- [Sandbox CLI](https://github.com/viswatejam702-tech/inspect_ai/blob/main/src/inspect_sandbox_tools/src/inspect_sandbox_tools/_cli/main.py): `src/inspect_sandbox_tools/src/inspect_sandbox_tools/_cli/main.py`
- [Sandbox server](https://github.com/viswatejam702-tech/inspect_ai/blob/main/src/inspect_sandbox_tools/src/inspect_sandbox_tools/_cli/server.py): `src/inspect_sandbox_tools/src/inspect_sandbox_tools/_cli/server.py`
- [Agent bridge](https://github.com/viswatejam702-tech/inspect_ai/blob/main/src/inspect_sandbox_tools/src/inspect_sandbox_tools/_agent_bridge/proxy.py): `src/inspect_sandbox_tools/src/inspect_sandbox_tools/_agent_bridge/proxy.py`
- [Remote bash](https://github.com/viswatejam702-tech/inspect_ai/blob/main/src/inspect_sandbox_tools/src/inspect_sandbox_tools/_remote_tools/_bash_session/_controller.py): `src/inspect_sandbox_tools/src/inspect_sandbox_tools/_remote_tools/_bash_session/_controller.py`
- [Remote execution](https://github.com/viswatejam702-tech/inspect_ai/blob/main/src/inspect_sandbox_tools/src/inspect_sandbox_tools/_remote_tools/_exec_remote/_controller.py): `src/inspect_sandbox_tools/src/inspect_sandbox_tools/_remote_tools/_exec_remote/_controller.py`
- [Remote MCP](https://github.com/viswatejam702-tech/inspect_ai/blob/main/src/inspect_sandbox_tools/src/inspect_sandbox_tools/_remote_tools/_mcp/mcp_server_session.py): `src/inspect_sandbox_tools/src/inspect_sandbox_tools/_remote_tools/_mcp/mcp_server_session.py`

### Other components

- **Evaluation author**

## Connections

- Evaluation author → CLI dispatch: launches
- CLI dispatch → Eval command: dispatches (evidence: `src/inspect_ai/_cli/main.py`)
- Eval command → Evaluation runner: starts
- Evaluation runner → Task execution: executes
- Task execution → Datasets: loads samples
- Task execution → Model interface: requests completion
- Task execution → Agent framework: runs agent
- Task execution → Scoring: scores results
- Model interface → Chat messages: uses messages
- Agent framework → Human approval: requests review
- ACP server → ACP connection: handles connections (evidence: `src/inspect_ai/agent/_acp/server.py`)
- ACP connection → ACP guards: uses guards (evidence: `src/inspect_ai/agent/_acp/connection.py`)
- ACP server → Discovery utilities: uses discovery (evidence: `src/inspect_ai/agent/_acp/server.py`)
- ACP server → Socket utilities: uses sockets (evidence: `src/inspect_ai/agent/_acp/server.py`)
- Session forwarder → ACP transport: forwards updates (evidence: `src/inspect_ai/agent/_acp/session_router.py`)
- Evaluation runner → Live sample state: reports live state
- Evaluation runner → Evaluation logs: records results
- Evaluation runner → Control server: hosts control
- Evaluation logs → Log analysis: feeds analysis
- Evaluation logs → Web viewer: shown in viewer
- CLI dispatch → Textual display: offers display
- Sandbox CLI → Agent bridge: dispatches proxy (evidence: `src/inspect_sandbox_tools/src/inspect_sandbox_tools/_cli/main.py`)
- Sandbox CLI → Sandbox server: starts server (evidence: `src/inspect_sandbox_tools/src/inspect_sandbox_tools/_cli/main.py`)
- Sandbox server → Remote bash: registers methods (evidence: `src/inspect_sandbox_tools/src/inspect_sandbox_tools/_cli/server.py`)
- Sandbox server → Remote execution: registers methods (evidence: `src/inspect_sandbox_tools/src/inspect_sandbox_tools/_cli/server.py`)
- Sandbox server → Remote MCP: registers methods (evidence: `src/inspect_sandbox_tools/src/inspect_sandbox_tools/_cli/server.py`)