# ethz-spylab/agentdojo architecture

> Architecture diagram of the GitHub repository ethz-spylab/agentdojo, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-09-25). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/ethz-spylab/agentdojo
- Repository: https://github.com/ethz-spylab/agentdojo

## Overview

AgentDojo is a Python benchmark for evaluating prompt-injection attacks and defenses in LLM agents. A researcher selects a benchmark suite, tasks, model, attack, and defense; the agent pipeline processes the task and invokes simulated tools against a suite environment. Task definitions provide ground-truth calls and evaluate task utility or injection security from outputs and environment state. The README documents these benchmark capabilities; sampled source confirms pipeline execution and task evaluation contracts, but not the full benchmark runner wiring.

Main entry points: [`src/agentdojo/agent_pipeline/agent_pipeline.py`](https://github.com/ethz-spylab/agentdojo/blob/main/src/agentdojo/agent_pipeline/agent_pipeline.py) (Agent pipeline), [`src/agentdojo/base_tasks.py`](https://github.com/ethz-spylab/agentdojo/blob/main/src/agentdojo/base_tasks.py) (Utility and security checks), [`src/agentdojo/default_suites`](https://github.com/ethz-spylab/agentdojo/tree/main/src/agentdojo/default_suites) (Suite definitions), [`src/agentdojo/functions_runtime.py`](https://github.com/ethz-spylab/agentdojo/blob/main/src/agentdojo/functions_runtime.py) (Functions runtime)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_orchestration["Agent orchestration"]
  node_pipeline["Agent pipeline<br/>[agent_pipeline.py]"]
  node_pipeline_elements["Pipeline elements<br/>[basic_elements.py]"]
  node_llm_adapters["LLM adapters"]
  node_tool_execution["Tool execution<br/>[tool_execution.py]"]
end

subgraph group_benchmark["Benchmark tasks"]
  node_suite_loader["Suite loader<br/>[load_suites.py]"]
  node_task_suite["Suite registry<br/>[task_suite.py]"]
  node_user_tasks["User tasks<br/>[base_tasks.py]"]
  node_injection_tasks["Injection tasks<br/>[base_tasks.py]"]
  node_suite_definitions["Suite definitions"]
end

subgraph group_environment["Simulated environment"]
  node_runtime["Functions runtime"]
  node_simulated_tools["Simulated tools"]
  node_environment_state[("Suite environments")]
end

subgraph group_security["Attack and defense"]
  node_attacks["Attack strategies"]
  node_pi_detector["Injection detector<br/>[pi_detector.py]"]
end

subgraph group_evaluation["Evaluation"]
  node_ground_truth["Ground-truth pipeline"]
  node_task_evaluation["Utility and security checks<br/>[base_tasks.py]"]
end

node_researcher(("Researcher"))
node_model_provider(("Model provider"))

node_researcher -->|"selects suite"| node_suite_loader
node_suite_loader -->|"loads suites"| node_task_suite
node_task_suite -->|"registers tasks"| node_suite_definitions
node_suite_definitions -->|"defines tasks"| node_user_tasks
node_suite_definitions -->|"defines attacks"| node_injection_tasks
node_researcher -->|"configures run"| node_pipeline
node_pipeline -->|"runs elements"| node_pipeline_elements
node_pipeline -->|"selects model"| node_llm_adapters
node_llm_adapters -->|"requests completion"| node_model_provider
node_pipeline_elements -->|"dispatches calls"| node_tool_execution
node_tool_execution -->|"runs function"| node_runtime
node_runtime -->|"dispatches tool"| node_simulated_tools
node_simulated_tools -->|"reads and updates"| node_environment_state
node_researcher -->|"selects attack"| node_attacks
node_attacks -->|"supplies attack input"| node_pipeline
node_pipeline -.->|"applies defense"| node_pi_detector
node_ground_truth -->|"gets expected calls"| node_user_tasks
node_ground_truth -->|"executes expected calls"| node_runtime
node_pipeline -->|"provides output"| node_task_evaluation
node_user_tasks -->|"checks utility"| node_task_evaluation
node_injection_tasks -->|"checks security"| node_task_evaluation
node_task_evaluation -->|"compares state"| node_environment_state

click node_pipeline "https://github.com/ethz-spylab/agentdojo/blob/main/src/agentdojo/agent_pipeline/agent_pipeline.py"
click node_pipeline_elements "https://github.com/ethz-spylab/agentdojo/blob/main/src/agentdojo/agent_pipeline/basic_elements.py"
click node_llm_adapters "https://github.com/ethz-spylab/agentdojo/tree/main/src/agentdojo/agent_pipeline/llms"
click node_tool_execution "https://github.com/ethz-spylab/agentdojo/blob/main/src/agentdojo/agent_pipeline/tool_execution.py"
click node_runtime "https://github.com/ethz-spylab/agentdojo/blob/main/src/agentdojo/functions_runtime.py"
click node_suite_loader "https://github.com/ethz-spylab/agentdojo/blob/main/src/agentdojo/task_suite/load_suites.py"
click node_task_suite "https://github.com/ethz-spylab/agentdojo/blob/main/src/agentdojo/task_suite/task_suite.py"
click node_user_tasks "https://github.com/ethz-spylab/agentdojo/blob/main/src/agentdojo/base_tasks.py"
click node_injection_tasks "https://github.com/ethz-spylab/agentdojo/blob/main/src/agentdojo/base_tasks.py"
click node_suite_definitions "https://github.com/ethz-spylab/agentdojo/tree/main/src/agentdojo/default_suites"
click node_simulated_tools "https://github.com/ethz-spylab/agentdojo/tree/main/src/agentdojo/default_suites/v1/tools"
click node_environment_state "https://github.com/ethz-spylab/agentdojo/tree/main/src/agentdojo/default_suites/v1"
click node_attacks "https://github.com/ethz-spylab/agentdojo/tree/main/src/agentdojo/attacks"
click node_pi_detector "https://github.com/ethz-spylab/agentdojo/blob/main/src/agentdojo/agent_pipeline/pi_detector.py"
click node_ground_truth "https://github.com/ethz-spylab/agentdojo/blob/main/src/agentdojo/agent_pipeline/ground_truth_pipeline.py"
click node_task_evaluation "https://github.com/ethz-spylab/agentdojo/blob/main/src/agentdojo/base_tasks.py"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_pipeline,node_pipeline_elements,node_llm_adapters,node_tool_execution toneBlue
class node_suite_loader,node_task_suite,node_user_tasks,node_injection_tasks,node_suite_definitions toneAmber
class node_runtime,node_simulated_tools,node_environment_state toneMint
class node_attacks,node_pi_detector toneRose
class node_ground_truth,node_task_evaluation,node_researcher,node_model_provider toneIndigo
```

## Components

### Agent orchestration

- [Agent pipeline](https://github.com/ethz-spylab/agentdojo/blob/main/src/agentdojo/agent_pipeline/agent_pipeline.py): `src/agentdojo/agent_pipeline/agent_pipeline.py`
- [Pipeline elements](https://github.com/ethz-spylab/agentdojo/blob/main/src/agentdojo/agent_pipeline/basic_elements.py): `src/agentdojo/agent_pipeline/basic_elements.py`
- [LLM adapters](https://github.com/ethz-spylab/agentdojo/tree/main/src/agentdojo/agent_pipeline/llms): `src/agentdojo/agent_pipeline/llms`
- [Tool execution](https://github.com/ethz-spylab/agentdojo/blob/main/src/agentdojo/agent_pipeline/tool_execution.py): `src/agentdojo/agent_pipeline/tool_execution.py`

### Benchmark tasks

- [Suite loader](https://github.com/ethz-spylab/agentdojo/blob/main/src/agentdojo/task_suite/load_suites.py): `src/agentdojo/task_suite/load_suites.py`
- [Suite registry](https://github.com/ethz-spylab/agentdojo/blob/main/src/agentdojo/task_suite/task_suite.py): `src/agentdojo/task_suite/task_suite.py`
- [User tasks](https://github.com/ethz-spylab/agentdojo/blob/main/src/agentdojo/base_tasks.py): `src/agentdojo/base_tasks.py`
- [Injection tasks](https://github.com/ethz-spylab/agentdojo/blob/main/src/agentdojo/base_tasks.py): `src/agentdojo/base_tasks.py`
- [Suite definitions](https://github.com/ethz-spylab/agentdojo/tree/main/src/agentdojo/default_suites): `src/agentdojo/default_suites`

### Simulated environment

- [Functions runtime](https://github.com/ethz-spylab/agentdojo/blob/main/src/agentdojo/functions_runtime.py): `src/agentdojo/functions_runtime.py`
- [Simulated tools](https://github.com/ethz-spylab/agentdojo/tree/main/src/agentdojo/default_suites/v1/tools): `src/agentdojo/default_suites/v1/tools`
- [Suite environments](https://github.com/ethz-spylab/agentdojo/tree/main/src/agentdojo/default_suites/v1): `src/agentdojo/default_suites/v1`

### Attack and defense

- [Attack strategies](https://github.com/ethz-spylab/agentdojo/tree/main/src/agentdojo/attacks): `src/agentdojo/attacks`
- [Injection detector](https://github.com/ethz-spylab/agentdojo/blob/main/src/agentdojo/agent_pipeline/pi_detector.py): `src/agentdojo/agent_pipeline/pi_detector.py`

### Evaluation

- [Ground-truth pipeline](https://github.com/ethz-spylab/agentdojo/blob/main/src/agentdojo/agent_pipeline/ground_truth_pipeline.py): `src/agentdojo/agent_pipeline/ground_truth_pipeline.py`
- [Utility and security checks](https://github.com/ethz-spylab/agentdojo/blob/main/src/agentdojo/base_tasks.py): `src/agentdojo/base_tasks.py`

### Other components

- **Researcher**
- **Model provider**

## Connections

- Researcher → Suite loader: selects suite
- Suite loader → Suite registry: loads suites
- Suite registry → Suite definitions: registers tasks
- Suite definitions → User tasks: defines tasks
- Suite definitions → Injection tasks: defines attacks
- Researcher → Agent pipeline: configures run
- Agent pipeline → Pipeline elements: runs elements
- Agent pipeline → LLM adapters: selects model
- LLM adapters → Model provider: requests completion
- Pipeline elements → Tool execution: dispatches calls
- Tool execution → Functions runtime: runs function
- Functions runtime → Simulated tools: dispatches tool
- Simulated tools → Suite environments: reads and updates
- Researcher → Attack strategies: selects attack
- Attack strategies → Agent pipeline: supplies attack input
- Agent pipeline → Injection detector: applies defense
- Ground-truth pipeline → User tasks: gets expected calls
- Ground-truth pipeline → Functions runtime: executes expected calls
- Agent pipeline → Utility and security checks: provides output
- User tasks → Utility and security checks: checks utility
- Injection tasks → Utility and security checks: checks security
- Utility and security checks → Suite environments: compares state