# antgroup/agent3sigma-canary architecture

> Architecture diagram of the GitHub repository antgroup/agent3sigma-canary, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-09-29). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/antgroup/agent3sigma-canary
- Repository: https://github.com/antgroup/agent3sigma-canary

## Overview

Agent3σ-Canary evaluates AI agents by running realistic tool-using tasks in isolated environments and grading their execution trajectories. The main path is task selection and attack setup, sandboxed agent execution against simulated resources, then multidimensional scoring and result analysis. The README also documents multiple agent frameworks, safety tasks, attack strategies, and security-defense variants. Some orchestration and task-corpus source is not present in the supplied tree, so those links use supported directory paths; unsampled implementation details are omitted.

Main entry points: [`scripts`](https://github.com/antgroup/agent3sigma-canary/tree/master/scripts) (Evaluation runner), [`scripts/lib_agent.py`](https://github.com/antgroup/agent3sigma-canary/blob/master/scripts/lib_agent.py) (Agent execution), [`workflow/images`](https://github.com/antgroup/agent3sigma-canary/tree/master/workflow/images) (Agent frameworks), [`scripts/lib_grading.py`](https://github.com/antgroup/agent3sigma-canary/blob/master/scripts/lib_grading.py) (Trajectory grading)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_evaluation["Evaluation orchestration"]
  node_runner["Evaluation runner"]
  node_tasks["Task suites<br/>[lib_tasks.py]"]
  node_attacks["Attack methods<br/>[lib_attacks.py]"]
end

subgraph group_runtime["Sandbox runtime"]
  node_agent["Agent execution<br/>[lib_agent.py]"]
  node_sandbox["Docker sandbox<br/>[lib_docker.py]"]
  node_frameworks["Agent frameworks"]
  node_hermes["Hermes shim<br/>[app.py]"]
  node_nanoclawshim["NanoClaw shims<br/>[driver.mjs]"]
  node_openai_shim["OpenAI shim<br/>[driver_openai.mjs]"]
end

subgraph group_defenses["Defense variants"]
  node_defense["Security defenses"]
  node_shield["OpenClaw Shield<br/>[index.ts]"]
  node_secureclaw["SecureClaw<br/>[index.ts]"]
  node_clawkeeper["ClawKeeper<br/>[index.js]"]
end

subgraph group_scoring["Scoring and analysis"]
  node_trajectory["Execution trajectory<br/>[lib_tracee.py]"]
  node_grading["Trajectory grading<br/>[lib_grading.py]"]
  node_tracegrading["Trace correlation"]
  node_results[("Evaluation results")]
  node_analyzer["Results analyzer<br/>[app.py]"]
end

subgraph group_resources["Simulated resources"]
  node_mockapi["Mock API<br/>[app.py]"]
  node_skills["Tool scenarios"]
end

node_user(("Evaluator"))
node_auxmodels(("Auxiliary models"))

node_user -->|"starts evaluation"| node_runner
node_runner -->|"selects tasks"| node_tasks
node_runner -->|"configures attacks"| node_attacks
node_runner -->|"launches sandbox"| node_sandbox
node_runner -->|"dispatches run"| node_agent
node_agent -->|"runs agent"| node_frameworks
node_frameworks -.->|"uses adapter"| node_hermes
node_frameworks -.->|"uses adapter"| node_nanoclawshim
node_frameworks -.->|"uses adapter"| node_openai_shim
node_agent -->|"calls simulated APIs"| node_mockapi
node_mockapi -->|"serves scenarios"| node_skills
node_agent -->|"records execution"| node_trajectory
node_trajectory -->|"provides trace"| node_tracegrading
node_runner -->|"requests scoring"| node_grading
node_grading -.->|"requests judgments"| node_auxmodels
node_grading -->|"writes scores"| node_results
node_runner -->|"stores run data"| node_results
node_analyzer -->|"reads results"| node_results
node_runner -.->|"selects variant"| node_defense
node_defense -.->|"may include"| node_shield
node_defense -.->|"may include"| node_secureclaw
node_defense -.->|"may include"| node_clawkeeper

click node_runner "https://github.com/antgroup/agent3sigma-canary/tree/master/scripts"
click node_tasks "https://github.com/antgroup/agent3sigma-canary/blob/master/scripts/lib_tasks.py"
click node_attacks "https://github.com/antgroup/agent3sigma-canary/blob/master/scripts/lib_attacks.py"
click node_agent "https://github.com/antgroup/agent3sigma-canary/blob/master/scripts/lib_agent.py"
click node_sandbox "https://github.com/antgroup/agent3sigma-canary/blob/master/scripts/lib_docker.py"
click node_frameworks "https://github.com/antgroup/agent3sigma-canary/tree/master/workflow/images"
click node_hermes "https://github.com/antgroup/agent3sigma-canary/blob/master/workflow/images/hermes/mock-api/app.py"
click node_nanoclawshim "https://github.com/antgroup/agent3sigma-canary/blob/master/workflow/images/nanoclaw/shim/driver.mjs"
click node_openai_shim "https://github.com/antgroup/agent3sigma-canary/blob/master/workflow/images/nanoclaw_openai/shim/driver_openai.mjs"
click node_mockapi "https://github.com/antgroup/agent3sigma-canary/blob/master/workflow/images/official/mock-api/app.py"
click node_skills "https://github.com/antgroup/agent3sigma-canary/tree/master/_skills_repository"
click node_trajectory "https://github.com/antgroup/agent3sigma-canary/blob/master/scripts/lib_tracee.py"
click node_grading "https://github.com/antgroup/agent3sigma-canary/blob/master/scripts/lib_grading.py"
click node_tracegrading "https://github.com/antgroup/agent3sigma-canary/blob/master/scripts/lib_tracee_grading.py"
click node_results "https://github.com/antgroup/agent3sigma-canary/tree/master/scripts"
click node_analyzer "https://github.com/antgroup/agent3sigma-canary/blob/master/workflow_analysis/app.py"
click node_defense "https://github.com/antgroup/agent3sigma-canary/tree/master/workflow/images"
click node_shield "https://github.com/antgroup/agent3sigma-canary/blob/master/workflow/images/official_shield/openclaw-shield/src/index.ts"
click node_secureclaw "https://github.com/antgroup/agent3sigma-canary/blob/master/workflow/images/official_secureclaw/secureclaw/secureclaw/src/index.ts"
click node_clawkeeper "https://github.com/antgroup/agent3sigma-canary/blob/master/workflow/images/official_clawkeeper/ClawKeeper/clawkeeper-plugin/src/index.js"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_runner,node_tasks,node_attacks toneBlue
class node_agent,node_sandbox,node_frameworks,node_hermes,node_nanoclawshim,node_openai_shim toneAmber
class node_defense,node_shield,node_secureclaw,node_clawkeeper toneMint
class node_trajectory,node_grading,node_tracegrading,node_results,node_analyzer toneRose
class node_mockapi,node_skills,node_user,node_auxmodels toneIndigo
```

## Components

### Evaluation orchestration

- [Evaluation runner](https://github.com/antgroup/agent3sigma-canary/tree/master/scripts): `scripts`
- [Task suites](https://github.com/antgroup/agent3sigma-canary/blob/master/scripts/lib_tasks.py): `scripts/lib_tasks.py`
- [Attack methods](https://github.com/antgroup/agent3sigma-canary/blob/master/scripts/lib_attacks.py): `scripts/lib_attacks.py`

### Sandbox runtime

- [Agent execution](https://github.com/antgroup/agent3sigma-canary/blob/master/scripts/lib_agent.py): `scripts/lib_agent.py`
- [Docker sandbox](https://github.com/antgroup/agent3sigma-canary/blob/master/scripts/lib_docker.py): `scripts/lib_docker.py`
- [Agent frameworks](https://github.com/antgroup/agent3sigma-canary/tree/master/workflow/images): `workflow/images`
- [Hermes shim](https://github.com/antgroup/agent3sigma-canary/blob/master/workflow/images/hermes/mock-api/app.py): `workflow/images/hermes/mock-api/app.py`
- [NanoClaw shims](https://github.com/antgroup/agent3sigma-canary/blob/master/workflow/images/nanoclaw/shim/driver.mjs): `workflow/images/nanoclaw/shim/driver.mjs`
- [OpenAI shim](https://github.com/antgroup/agent3sigma-canary/blob/master/workflow/images/nanoclaw_openai/shim/driver_openai.mjs): `workflow/images/nanoclaw_openai/shim/driver_openai.mjs`

### Defense variants

- [Security defenses](https://github.com/antgroup/agent3sigma-canary/tree/master/workflow/images): `workflow/images`
- [OpenClaw Shield](https://github.com/antgroup/agent3sigma-canary/blob/master/workflow/images/official_shield/openclaw-shield/src/index.ts): `workflow/images/official_shield/openclaw-shield/src/index.ts`
- [SecureClaw](https://github.com/antgroup/agent3sigma-canary/blob/master/workflow/images/official_secureclaw/secureclaw/secureclaw/src/index.ts): `workflow/images/official_secureclaw/secureclaw/secureclaw/src/index.ts`
- [ClawKeeper](https://github.com/antgroup/agent3sigma-canary/blob/master/workflow/images/official_clawkeeper/ClawKeeper/clawkeeper-plugin/src/index.js): `workflow/images/official_clawkeeper/ClawKeeper/clawkeeper-plugin/src/index.js`

### Scoring and analysis

- [Execution trajectory](https://github.com/antgroup/agent3sigma-canary/blob/master/scripts/lib_tracee.py): `scripts/lib_tracee.py`
- [Trajectory grading](https://github.com/antgroup/agent3sigma-canary/blob/master/scripts/lib_grading.py): `scripts/lib_grading.py`
- [Trace correlation](https://github.com/antgroup/agent3sigma-canary/blob/master/scripts/lib_tracee_grading.py): `scripts/lib_tracee_grading.py`
- [Evaluation results](https://github.com/antgroup/agent3sigma-canary/tree/master/scripts): `scripts`
- [Results analyzer](https://github.com/antgroup/agent3sigma-canary/blob/master/workflow_analysis/app.py): `workflow_analysis/app.py`

### Simulated resources

- [Mock API](https://github.com/antgroup/agent3sigma-canary/blob/master/workflow/images/official/mock-api/app.py): `workflow/images/official/mock-api/app.py`
- [Tool scenarios](https://github.com/antgroup/agent3sigma-canary/tree/master/_skills_repository): `_skills_repository`

### Other components

- **Evaluator**
- **Auxiliary models**

## Connections

- Evaluator → Evaluation runner: starts evaluation
- Evaluation runner → Task suites: selects tasks
- Evaluation runner → Attack methods: configures attacks
- Evaluation runner → Docker sandbox: launches sandbox
- Evaluation runner → Agent execution: dispatches run
- Agent execution → Agent frameworks: runs agent
- Agent frameworks → Hermes shim: uses adapter
- Agent frameworks → NanoClaw shims: uses adapter
- Agent frameworks → OpenAI shim: uses adapter
- Agent execution → Mock API: calls simulated APIs
- Mock API → Tool scenarios: serves scenarios
- Agent execution → Execution trajectory: records execution
- Execution trajectory → Trace correlation: provides trace
- Evaluation runner → Trajectory grading: requests scoring
- Trajectory grading → Auxiliary models: requests judgments
- Trajectory grading → Evaluation results: writes scores
- Evaluation runner → Evaluation results: stores run data
- Results analyzer → Evaluation results: reads results
- Evaluation runner → Security defenses: selects variant
- Security defenses → OpenClaw Shield: may include
- Security defenses → SecureClaw: may include
- Security defenses → ClawKeeper: may include