# skygod07/ai-agent-security---multi-step-tool-attacks architecture

> Architecture diagram of the GitHub repository skygod07/ai-agent-security---multi-step-tool-attacks, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-10-03). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/skygod07/ai-agent-security---multi-step-tool-attacks
- Repository: https://github.com/skygod07/ai-agent-security---multi-step-tool-attacks

## Overview

This repository combines a Kaggle AI-agent security submission pipeline with a reusable SDK for attack algorithms, sandboxed tool interactions, guardrails, and evaluation. The main workflow is to implement or discover multi-step attack candidates, run them through an agent and tool environment, then score and inspect results through evaluation commands or the Kaggle gateway. It also contains a separate ROS visual-editor application, omitted here because the README identifies the competition pipeline as the product. Some subsystem links reflect documented capability or source-index references; unsampled implementation details are not inferred.

Main entry points: [`our_work/aicomp_sdk/evaluation/ops.py`](https://github.com/skygod07/ai-agent-security---multi-step-tool-attacks/blob/main/our_work/aicomp_sdk/evaluation/ops.py) (Evaluation Operations), [`our_work/kaggle_evaluation/jed_attack_134815/jed_attack_gateway.py`](https://github.com/skygod07/ai-agent-security---multi-step-tool-attacks/blob/main/our_work/kaggle_evaluation/jed_attack_134815/jed_attack_gateway.py) (Competition Gateway), [`our_work/src/attack_discovery/search.py`](https://github.com/skygod07/ai-agent-security---multi-step-tool-attacks/blob/main/our_work/src/attack_discovery/search.py) (Search Strategies), [`our_work/src/attack_discovery/candidate.py`](https://github.com/skygod07/ai-agent-security---multi-step-tool-attacks/blob/main/our_work/src/attack_discovery/candidate.py) (Candidate Model)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_discovery["Attack Discovery"]
  node_search["Search Strategies<br/>[search.py]"]
  node_candidate["Candidate Model<br/>[candidate.py]"]
  node_mutation["Candidate Mutation<br/>[mutation.py]"]
  node_trace["Trace Analysis<br/>[trace.py]"]
  node_stockfish["Attack Search Engine"]
end

subgraph group_runtime["Agent Runtime"]
  node_contracts["Attack Contracts<br/>[contracts.py]"]
  node_agents["Agent Backends"]
  node_tools["Tool Suite"]
  node_cells["Trace Cells<br/>[cells.py]"]
  node_predicates["Security Predicates<br/>[predicates.py]"]
  node_guardrails["Guardrails<br/>[optimal.py]"]
  node_hooks["Runtime Hooks"]
end

subgraph group_evaluation["Evaluation"]
  node_evalops["Evaluation Operations<br/>[ops.py]"]
  node_diagnostics["Run Diagnostics"]
  node_scoring["Scoring<br/>[scoring.py]"]
end

subgraph group_submission["Submission Interface"]
  node_cli["SDK CLI<br/>[main.py]"]
  node_evalcommand["Evaluate Command<br/>[evaluate.py]"]
  node_kaggle_gateway["Competition Gateway"]
  node_inference["Inference Server"]
  node_remote_agent["Remote Agent<br/>[remote_agent.py]"]
  node_base_gateway["Gateway Base<br/>[base_gateway.py]"]
  node_relay["Evaluation Relay<br/>[relay.py]"]
  node_templates["Gateway Templates<br/>[templates.py]"]
end

node_researcher(("Researcher"))

node_researcher -->|"runs commands"| node_cli
node_cli -->|"dispatches evaluation"| node_evalcommand
node_evalcommand -.->|"runs evaluation"| node_evalops
node_evalops -->|"loads attack algorithms"| node_contracts
node_evalops -->|"checks traces"| node_predicates
node_evalops -->|"signs tool events"| node_cells
node_evalops -->|"uses guardrails"| node_guardrails
node_evalops -->|"loads hooks"| node_hooks
node_evalops -->|"scores findings"| node_scoring
node_evalops -->|"emits diagnostics"| node_diagnostics
node_search -->|"evaluates candidates"| node_candidate
node_search -->|"explores mutations"| node_mutation
node_search -->|"analyzes traces"| node_trace
node_candidate -->|"uses attack contracts"| node_contracts
node_candidate -->|"checks predicates"| node_predicates
node_stockfish -->|"evaluates traces"| node_trace
node_kaggle_gateway -->|"extends gateway"| node_base_gateway
node_kaggle_gateway -->|"runs evaluation"| node_evalops
node_kaggle_gateway -->|"uses agent backends"| node_agents
node_kaggle_gateway -->|"uses guardrails"| node_guardrails
node_kaggle_gateway -->|"dispatches agent"| node_remote_agent
node_kaggle_gateway -->|"uses templates"| node_templates
node_inference -->|"uses gateway"| node_kaggle_gateway
node_inference -->|"runs evaluation"| node_evalops
node_base_gateway -->|"uses relay"| node_relay

click node_search "https://github.com/skygod07/ai-agent-security---multi-step-tool-attacks/blob/main/our_work/src/attack_discovery/search.py"
click node_candidate "https://github.com/skygod07/ai-agent-security---multi-step-tool-attacks/blob/main/our_work/src/attack_discovery/candidate.py"
click node_mutation "https://github.com/skygod07/ai-agent-security---multi-step-tool-attacks/blob/main/our_work/src/attack_discovery/mutation.py"
click node_trace "https://github.com/skygod07/ai-agent-security---multi-step-tool-attacks/blob/main/our_work/src/attack_discovery/trace.py"
click node_stockfish "https://github.com/skygod07/ai-agent-security---multi-step-tool-attacks/blob/main/our_work/src/attack_discovery/stockfish_engine.py"
click node_contracts "https://github.com/skygod07/ai-agent-security---multi-step-tool-attacks/blob/main/our_work/aicomp_sdk/attacks/contracts.py"
click node_agents "https://github.com/skygod07/ai-agent-security---multi-step-tool-attacks/tree/main/our_work/aicomp_sdk/agents"
click node_tools "https://github.com/skygod07/ai-agent-security---multi-step-tool-attacks/tree/main/our_work/aicomp_sdk/core/tools"
click node_cells "https://github.com/skygod07/ai-agent-security---multi-step-tool-attacks/blob/main/our_work/aicomp_sdk/core/cells.py"
click node_predicates "https://github.com/skygod07/ai-agent-security---multi-step-tool-attacks/blob/main/our_work/aicomp_sdk/core/predicates.py"
click node_guardrails "https://github.com/skygod07/ai-agent-security---multi-step-tool-attacks/blob/main/our_work/aicomp_sdk/guardrails/optimal.py"
click node_hooks "https://github.com/skygod07/ai-agent-security---multi-step-tool-attacks/tree/main/our_work/aicomp_sdk/hooks"
click node_evalops "https://github.com/skygod07/ai-agent-security---multi-step-tool-attacks/blob/main/our_work/aicomp_sdk/evaluation/ops.py"
click node_diagnostics "https://github.com/skygod07/ai-agent-security---multi-step-tool-attacks/tree/main/our_work/aicomp_sdk/evaluation/diagnostics"
click node_scoring "https://github.com/skygod07/ai-agent-security---multi-step-tool-attacks/blob/main/our_work/aicomp_sdk/scoring.py"
click node_cli "https://github.com/skygod07/ai-agent-security---multi-step-tool-attacks/blob/main/our_work/aicomp_sdk/cli/main.py"
click node_evalcommand "https://github.com/skygod07/ai-agent-security---multi-step-tool-attacks/blob/main/our_work/aicomp_sdk/cli/commands/evaluate.py"
click node_kaggle_gateway "https://github.com/skygod07/ai-agent-security---multi-step-tool-attacks/blob/main/our_work/kaggle_evaluation/jed_attack_134815/jed_attack_gateway.py"
click node_inference "https://github.com/skygod07/ai-agent-security---multi-step-tool-attacks/blob/main/our_work/kaggle_evaluation/jed_attack_134815/jed_attack_inference_server.py"
click node_remote_agent "https://github.com/skygod07/ai-agent-security---multi-step-tool-attacks/blob/main/our_work/kaggle_evaluation/jed_attack_134815/remote_agent.py"
click node_base_gateway "https://github.com/skygod07/ai-agent-security---multi-step-tool-attacks/blob/main/our_work/kaggle_evaluation/core/base_gateway.py"
click node_relay "https://github.com/skygod07/ai-agent-security---multi-step-tool-attacks/blob/main/our_work/kaggle_evaluation/core/relay.py"
click node_templates "https://github.com/skygod07/ai-agent-security---multi-step-tool-attacks/blob/main/our_work/kaggle_evaluation/core/templates.py"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_search,node_candidate,node_mutation,node_trace,node_stockfish toneBlue
class node_contracts,node_agents,node_tools,node_cells,node_predicates,node_guardrails,node_hooks toneAmber
class node_evalops,node_diagnostics,node_scoring toneMint
class node_cli,node_evalcommand,node_kaggle_gateway,node_inference,node_remote_agent,node_base_gateway,node_relay,node_templates toneRose
class node_researcher toneIndigo
```

## Components

### Attack Discovery

- [Search Strategies](https://github.com/skygod07/ai-agent-security---multi-step-tool-attacks/blob/main/our_work/src/attack_discovery/search.py): `our_work/src/attack_discovery/search.py`
- [Candidate Model](https://github.com/skygod07/ai-agent-security---multi-step-tool-attacks/blob/main/our_work/src/attack_discovery/candidate.py): `our_work/src/attack_discovery/candidate.py`
- [Candidate Mutation](https://github.com/skygod07/ai-agent-security---multi-step-tool-attacks/blob/main/our_work/src/attack_discovery/mutation.py): `our_work/src/attack_discovery/mutation.py`
- [Trace Analysis](https://github.com/skygod07/ai-agent-security---multi-step-tool-attacks/blob/main/our_work/src/attack_discovery/trace.py): `our_work/src/attack_discovery/trace.py`
- [Attack Search Engine](https://github.com/skygod07/ai-agent-security---multi-step-tool-attacks/blob/main/our_work/src/attack_discovery/stockfish_engine.py): `our_work/src/attack_discovery/stockfish_engine.py`

### Agent Runtime

- [Attack Contracts](https://github.com/skygod07/ai-agent-security---multi-step-tool-attacks/blob/main/our_work/aicomp_sdk/attacks/contracts.py): `our_work/aicomp_sdk/attacks/contracts.py`
- [Agent Backends](https://github.com/skygod07/ai-agent-security---multi-step-tool-attacks/tree/main/our_work/aicomp_sdk/agents): `our_work/aicomp_sdk/agents`
- [Tool Suite](https://github.com/skygod07/ai-agent-security---multi-step-tool-attacks/tree/main/our_work/aicomp_sdk/core/tools): `our_work/aicomp_sdk/core/tools`
- [Trace Cells](https://github.com/skygod07/ai-agent-security---multi-step-tool-attacks/blob/main/our_work/aicomp_sdk/core/cells.py): `our_work/aicomp_sdk/core/cells.py`
- [Security Predicates](https://github.com/skygod07/ai-agent-security---multi-step-tool-attacks/blob/main/our_work/aicomp_sdk/core/predicates.py): `our_work/aicomp_sdk/core/predicates.py`
- [Guardrails](https://github.com/skygod07/ai-agent-security---multi-step-tool-attacks/blob/main/our_work/aicomp_sdk/guardrails/optimal.py): `our_work/aicomp_sdk/guardrails/optimal.py`
- [Runtime Hooks](https://github.com/skygod07/ai-agent-security---multi-step-tool-attacks/tree/main/our_work/aicomp_sdk/hooks): `our_work/aicomp_sdk/hooks`

### Evaluation

- [Evaluation Operations](https://github.com/skygod07/ai-agent-security---multi-step-tool-attacks/blob/main/our_work/aicomp_sdk/evaluation/ops.py): `our_work/aicomp_sdk/evaluation/ops.py`
- [Run Diagnostics](https://github.com/skygod07/ai-agent-security---multi-step-tool-attacks/tree/main/our_work/aicomp_sdk/evaluation/diagnostics): `our_work/aicomp_sdk/evaluation/diagnostics`
- [Scoring](https://github.com/skygod07/ai-agent-security---multi-step-tool-attacks/blob/main/our_work/aicomp_sdk/scoring.py): `our_work/aicomp_sdk/scoring.py`

### Submission Interface

- [SDK CLI](https://github.com/skygod07/ai-agent-security---multi-step-tool-attacks/blob/main/our_work/aicomp_sdk/cli/main.py): `our_work/aicomp_sdk/cli/main.py`
- [Evaluate Command](https://github.com/skygod07/ai-agent-security---multi-step-tool-attacks/blob/main/our_work/aicomp_sdk/cli/commands/evaluate.py): `our_work/aicomp_sdk/cli/commands/evaluate.py`
- [Competition Gateway](https://github.com/skygod07/ai-agent-security---multi-step-tool-attacks/blob/main/our_work/kaggle_evaluation/jed_attack_134815/jed_attack_gateway.py): `our_work/kaggle_evaluation/jed_attack_134815/jed_attack_gateway.py`
- [Inference Server](https://github.com/skygod07/ai-agent-security---multi-step-tool-attacks/blob/main/our_work/kaggle_evaluation/jed_attack_134815/jed_attack_inference_server.py): `our_work/kaggle_evaluation/jed_attack_134815/jed_attack_inference_server.py`
- [Remote Agent](https://github.com/skygod07/ai-agent-security---multi-step-tool-attacks/blob/main/our_work/kaggle_evaluation/jed_attack_134815/remote_agent.py): `our_work/kaggle_evaluation/jed_attack_134815/remote_agent.py`
- [Gateway Base](https://github.com/skygod07/ai-agent-security---multi-step-tool-attacks/blob/main/our_work/kaggle_evaluation/core/base_gateway.py): `our_work/kaggle_evaluation/core/base_gateway.py`
- [Evaluation Relay](https://github.com/skygod07/ai-agent-security---multi-step-tool-attacks/blob/main/our_work/kaggle_evaluation/core/relay.py): `our_work/kaggle_evaluation/core/relay.py`
- [Gateway Templates](https://github.com/skygod07/ai-agent-security---multi-step-tool-attacks/blob/main/our_work/kaggle_evaluation/core/templates.py): `our_work/kaggle_evaluation/core/templates.py`

### Other components

- **Researcher**

## Connections

- Researcher → SDK CLI: runs commands (evidence: `our_work/aicomp_sdk/cli/main.py`)
- SDK CLI → Evaluate Command: dispatches evaluation (evidence: `our_work/aicomp_sdk/cli/main.py`)
- Evaluate Command → Evaluation Operations: runs evaluation (evidence: `our_work/aicomp_sdk/cli/commands/evaluate.py`)
- Evaluation Operations → Attack Contracts: loads attack algorithms (evidence: `our_work/aicomp_sdk/evaluation/ops.py`)
- Evaluation Operations → Security Predicates: checks traces (evidence: `our_work/aicomp_sdk/evaluation/ops.py`)
- Evaluation Operations → Trace Cells: signs tool events (evidence: `our_work/aicomp_sdk/evaluation/ops.py`)
- Evaluation Operations → Guardrails: uses guardrails (evidence: `our_work/aicomp_sdk/evaluation/ops.py`)
- Evaluation Operations → Runtime Hooks: loads hooks (evidence: `our_work/aicomp_sdk/evaluation/ops.py`)
- Evaluation Operations → Scoring: scores findings (evidence: `our_work/aicomp_sdk/evaluation/ops.py`)
- Evaluation Operations → Run Diagnostics: emits diagnostics (evidence: `our_work/aicomp_sdk/evaluation/ops.py`)
- Search Strategies → Candidate Model: evaluates candidates (evidence: `our_work/src/attack_discovery/search.py`)
- Search Strategies → Candidate Mutation: explores mutations (evidence: `our_work/src/attack_discovery/search.py`)
- Search Strategies → Trace Analysis: analyzes traces (evidence: `our_work/src/attack_discovery/search.py`)
- Candidate Model → Attack Contracts: uses attack contracts (evidence: `our_work/src/attack_discovery/candidate.py`)
- Candidate Model → Security Predicates: checks predicates (evidence: `our_work/src/attack_discovery/candidate.py`)
- Attack Search Engine → Trace Analysis: evaluates traces (evidence: `our_work/src/attack_discovery/stockfish_engine.py`)
- Competition Gateway → Gateway Base: extends gateway (evidence: `our_work/kaggle_evaluation/jed_attack_134815/jed_attack_gateway.py`)
- Competition Gateway → Evaluation Operations: runs evaluation (evidence: `our_work/kaggle_evaluation/jed_attack_134815/jed_attack_gateway.py`)
- Competition Gateway → Agent Backends: uses agent backends (evidence: `our_work/kaggle_evaluation/jed_attack_134815/jed_attack_gateway.py`)
- Competition Gateway → Guardrails: uses guardrails (evidence: `our_work/kaggle_evaluation/jed_attack_134815/jed_attack_gateway.py`)
- Competition Gateway → Remote Agent: dispatches agent (evidence: `our_work/kaggle_evaluation/jed_attack_134815/jed_attack_gateway.py`)
- Competition Gateway → Gateway Templates: uses templates (evidence: `our_work/kaggle_evaluation/jed_attack_134815/jed_attack_gateway.py`)
- Inference Server → Competition Gateway: uses gateway (evidence: `our_work/kaggle_evaluation/jed_attack_134815/jed_attack_inference_server.py`)
- Inference Server → Evaluation Operations: runs evaluation (evidence: `our_work/kaggle_evaluation/jed_attack_134815/jed_attack_inference_server.py`)
- Gateway Base → Evaluation Relay: uses relay (evidence: `our_work/kaggle_evaluation/core/base_gateway.py`)
