# zixin22/fragfuse architecture

> Architecture diagram of the GitHub repository zixin22/fragfuse, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-10-02). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/zixin22/fragfuse
- Repository: https://github.com/zixin22/fragfuse

## Overview

FragFuse is a research suite for evaluating memory-based access-control risks in agents, organized into AgentHarm, RAP/WebShop, and OSAgent/SeeAct experiment tracks. The main workflow constructs or injects fragmented intent, runs an agent in a benchmark environment, and applies rule or guard-agent checks to assess unsafe behavior. The graph preserves the tracks’ distinct implementations rather than implying they share a runtime. Relationships are limited to README statements and source-index references or excerpts; several pipeline stages are documented but their internal calls were not sampled.

Main entry points: [`RAP/webshop/main.py`](https://github.com/zixin22/fragfuse/blob/main/RAP/webshop/main.py) (Attack runner), [`RAP/guard agent/webshop_guard_agent.py`](https://github.com/zixin22/fragfuse/blob/main/RAP/guard%20agent/webshop_guard_agent.py) (WebShop guard adapter), [`Agentharm/src/inspect_evals/agentharm/agentharm.py`](https://github.com/zixin22/fragfuse/blob/main/Agentharm/src/inspect_evals/agentharm/agentharm.py) (Benchmark task), [`OSAgent and SeeAct/AGrail4Agent/DAS/guardrail.py`](https://github.com/zixin22/fragfuse/blob/main/OSAgent%20and%20SeeAct/AGrail4Agent/DAS/guardrail.py) (Action guardrail)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_agentharm["AgentHarm evaluation"]
  node_ah_task["Benchmark task<br/>[agentharm.py]"]
  node_ah_agents["Agent variants"]
  node_ah_memory["Memory agent<br/>[memory_agent.py]"]
  node_ah_fragment["Fragment extraction"]
  node_ah_mask["Iterative masking"]
  node_ah_summaries["Fragment summaries"]
  node_ah_host["Host checks<br/>[4host_check.py]"]
  node_ah_injection["Memory injection"]
  node_ah_retrieval["Retrieval tests<br/>[8retrieve_rate.py]"]
  node_ah_guard["Guardrail prompts"]
  node_ah_scorer["Evaluation scoring<br/>[scorer.py]"]
end

subgraph group_rap["RAP WebShop"]
  node_rap_attack["Attack runner<br/>[main.py]"]
  node_rap_frag["Fragment construction"]
  node_rap_prompts["WebShop prompts<br/>[webshop_prompt.py]"]
  node_rap_rules["Rule checking<br/>[rule_checker.py]"]
  node_rap_profile["User profiles<br/>[user_profile.py]"]
  node_rap_request["Rule examples<br/>[request_webshop.py]"]
  node_rap_metrics["Rule metrics<br/>[metrics.py]"]
  node_rap_guard_adapter["WebShop guard adapter"]
  node_rap_guard["Guard agent<br/>[guardagent.py]"]
  node_rap_guard_prompts["Guard prompts<br/>[prompts_guard.py]"]
  node_rap_toolset["WebShop guard tools<br/>[toolset_webshop.py]"]
  node_rap_optimizer["Query optimization"]
  node_rap_template["Carrier templates"]
end

subgraph group_osseeact["OSAgent and SeeAct"]
  node_os_runner["OS experiment runner<br/>[exp_OS.py]"]
  node_os_guard["Action guardrail<br/>[guardrail.py]"]
  node_os_prompts["Guard prompts<br/>[prompts.py]"]
  node_os_permissions["Permission detection<br/>[permission_tool.py]"]
  node_os_tools["Tool interface<br/>[tool.py]"]
  node_os_web_tool["Web detection<br/>[web_tool.py]"]
  node_os_code_tool["Code detection<br/>[code_tool.py]"]
end

node_researcher(("Researcher"))
node_rap_user(("Benchmark operator"))
node_os_operator(("Experiment operator"))

node_researcher -->|"configures"| node_ah_task
node_ah_task -->|"selects agent"| node_ah_agents
node_ah_memory -->|"implements variant"| node_ah_agents
node_ah_memory -.->|"runs in task"| node_ah_task
node_ah_task -->|"scores results"| node_ah_scorer
node_ah_scorer -->|"uses metrics"| node_ah_task
node_ah_memory -->|"uses prompts"| node_ah_guard
node_researcher -->|"starts pipeline"| node_ah_fragment
node_ah_fragment -.->|"precedes masking"| node_ah_mask
node_ah_mask -.->|"prepares fragments"| node_ah_summaries
node_ah_summaries -.->|"feeds host checks"| node_ah_host
node_ah_host -.->|"precedes injection"| node_ah_injection
node_ah_injection -->|"followed by retrieval"| node_ah_retrieval
node_ah_retrieval -.->|"evaluates mitigation"| node_ah_guard
node_rap_user -->|"runs experiment"| node_rap_attack
node_rap_attack -.->|"uses attack inputs"| node_rap_frag
node_rap_attack -->|"uses prompts"| node_rap_prompts
node_rap_attack -->|"runs rule checks"| node_rap_rules
node_rap_attack -->|"selects defense"| node_rap_guard_adapter
node_rap_attack -.->|"evaluates outcomes"| node_rap_metrics
node_rap_rules -->|"checks profile"| node_rap_profile
node_rap_rules -.->|"uses rule examples"| node_rap_request
node_rap_guard_adapter -->|"delegates guard work"| node_rap_guard
node_rap_guard_adapter -->|"uses toolset"| node_rap_toolset
node_rap_guard_adapter -->|"uses profile data"| node_rap_profile
node_rap_guard_adapter -->|"uses rule examples"| node_rap_request
node_rap_guard -->|"loads prompts"| node_rap_guard_prompts
node_rap_toolset -->|"uses prompt definitions"| node_rap_guard_prompts
node_rap_user -.->|"optimizes queries"| node_rap_optimizer
node_rap_optimizer -->|"rewrites templates"| node_rap_template
node_rap_frag -.->|"builds carrier queries"| node_rap_template
node_os_operator -->|"runs experiment"| node_os_runner
node_os_runner -->|"uses prompts"| node_os_prompts
node_os_runner -->|"invokes guardrail"| node_os_guard
node_os_guard -->|"dispatches checks"| node_os_permissions
node_os_guard -->|"dispatches checks"| node_os_web_tool
node_os_guard -->|"dispatches checks"| node_os_code_tool
node_os_permissions -->|"inherits interface"| node_os_tools
node_os_web_tool -->|"inherits interface"| node_os_tools
node_os_code_tool -->|"inherits interface"| node_os_tools

click node_ah_task "https://github.com/zixin22/fragfuse/blob/main/Agentharm/src/inspect_evals/agentharm/agentharm.py"
click node_ah_agents "https://github.com/zixin22/fragfuse/tree/main/Agentharm/src/inspect_evals/agentharm/agents"
click node_ah_memory "https://github.com/zixin22/fragfuse/blob/main/Agentharm/src/inspect_evals/agentharm/agents/memory_agent.py"
click node_ah_fragment "https://github.com/zixin22/fragfuse/blob/main/Agentharm/src/inspect_evals/agentharm/fragfuse_pipeline/1extract_fragments.py"
click node_ah_mask "https://github.com/zixin22/fragfuse/blob/main/Agentharm/src/inspect_evals/agentharm/fragfuse_pipeline/2_1iterative_mask.py"
click node_ah_summaries "https://github.com/zixin22/fragfuse/tree/main/Agentharm/src/inspect_evals/agentharm/fragfuse_pipeline"
click node_ah_host "https://github.com/zixin22/fragfuse/blob/main/Agentharm/src/inspect_evals/agentharm/fragfuse_pipeline/4host_check.py"
click node_ah_injection "https://github.com/zixin22/fragfuse/blob/main/Agentharm/src/inspect_evals/agentharm/fragfuse_pipeline/5injection_store.py"
click node_ah_retrieval "https://github.com/zixin22/fragfuse/blob/main/Agentharm/src/inspect_evals/agentharm/fragfuse_pipeline/8retrieve_rate.py"
click node_ah_guard "https://github.com/zixin22/fragfuse/blob/main/Agentharm/src/inspect_evals/agentharm/guardrail_prompt/guardrail_prompt.py"
click node_ah_scorer "https://github.com/zixin22/fragfuse/blob/main/Agentharm/src/inspect_evals/agentharm/scorer.py"
click node_rap_attack "https://github.com/zixin22/fragfuse/blob/main/RAP/webshop/main.py"
click node_rap_frag "https://github.com/zixin22/fragfuse/blob/main/RAP/webshop/fragExtracor_pipeline/frag_mask_pipeline.py"
click node_rap_prompts "https://github.com/zixin22/fragfuse/blob/main/RAP/webshop/prompts/webshop_prompt.py"
click node_rap_rules "https://github.com/zixin22/fragfuse/blob/main/RAP/webshop/rule_and_profile/rule_checker.py"
click node_rap_profile "https://github.com/zixin22/fragfuse/blob/main/RAP/webshop/rule_and_profile/user_profile.py"
click node_rap_request "https://github.com/zixin22/fragfuse/blob/main/RAP/webshop/rule_and_profile/request_webshop.py"
click node_rap_metrics "https://github.com/zixin22/fragfuse/blob/main/RAP/webshop/rule_and_profile/metrics.py"
click node_rap_guard_adapter "https://github.com/zixin22/fragfuse/blob/main/RAP/guard%20agent/webshop_guard_agent.py"
click node_rap_guard "https://github.com/zixin22/fragfuse/blob/main/RAP/guard%20agent/guardagent.py"
click node_rap_guard_prompts "https://github.com/zixin22/fragfuse/blob/main/RAP/guard%20agent/prompts_guard.py"
click node_rap_toolset "https://github.com/zixin22/fragfuse/blob/main/RAP/guard%20agent/toolset_webshop.py"
click node_rap_optimizer "https://github.com/zixin22/fragfuse/tree/main/RAP/webshop/optimization/optimization_webshop"
click node_rap_template "https://github.com/zixin22/fragfuse/blob/main/RAP/webshop/optimization/optimization_webshop/carrier_query_template_utils.py"
click node_os_runner "https://github.com/zixin22/fragfuse/blob/main/OSAgent%20and%20SeeAct/AGrail4Agent/DAS/exp_OS.py"
click node_os_guard "https://github.com/zixin22/fragfuse/blob/main/OSAgent%20and%20SeeAct/AGrail4Agent/DAS/guardrail.py"
click node_os_prompts "https://github.com/zixin22/fragfuse/blob/main/OSAgent%20and%20SeeAct/AGrail4Agent/DAS/prompts.py"
click node_os_permissions "https://github.com/zixin22/fragfuse/blob/main/OSAgent%20and%20SeeAct/AGrail4Agent/DAS/tools/permission_tool.py"
click node_os_tools "https://github.com/zixin22/fragfuse/blob/main/OSAgent%20and%20SeeAct/AGrail4Agent/DAS/tools/tool.py"
click node_os_web_tool "https://github.com/zixin22/fragfuse/blob/main/OSAgent%20and%20SeeAct/AGrail4Agent/DAS/tools/web_tool.py"
click node_os_code_tool "https://github.com/zixin22/fragfuse/blob/main/OSAgent%20and%20SeeAct/AGrail4Agent/DAS/tools/code_tool.py"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_ah_task,node_ah_agents,node_ah_memory,node_ah_fragment,node_ah_mask,node_ah_summaries,node_ah_host,node_ah_injection,node_ah_retrieval,node_ah_guard,node_ah_scorer toneBlue
class node_rap_attack,node_rap_frag,node_rap_prompts,node_rap_rules,node_rap_profile,node_rap_request,node_rap_metrics,node_rap_guard_adapter,node_rap_guard,node_rap_guard_prompts,node_rap_toolset,node_rap_optimizer,node_rap_template toneAmber
class node_os_runner,node_os_guard,node_os_prompts,node_os_permissions,node_os_tools,node_os_web_tool,node_os_code_tool toneMint
class node_researcher,node_rap_user,node_os_operator toneIndigo
```

## Components

### AgentHarm evaluation

- [Benchmark task](https://github.com/zixin22/fragfuse/blob/main/Agentharm/src/inspect_evals/agentharm/agentharm.py): `Agentharm/src/inspect_evals/agentharm/agentharm.py`
- [Agent variants](https://github.com/zixin22/fragfuse/tree/main/Agentharm/src/inspect_evals/agentharm/agents): `Agentharm/src/inspect_evals/agentharm/agents`
- [Memory agent](https://github.com/zixin22/fragfuse/blob/main/Agentharm/src/inspect_evals/agentharm/agents/memory_agent.py): `Agentharm/src/inspect_evals/agentharm/agents/memory_agent.py`
- [Fragment extraction](https://github.com/zixin22/fragfuse/blob/main/Agentharm/src/inspect_evals/agentharm/fragfuse_pipeline/1extract_fragments.py): `Agentharm/src/inspect_evals/agentharm/fragfuse_pipeline/1extract_fragments.py`
- [Iterative masking](https://github.com/zixin22/fragfuse/blob/main/Agentharm/src/inspect_evals/agentharm/fragfuse_pipeline/2_1iterative_mask.py): `Agentharm/src/inspect_evals/agentharm/fragfuse_pipeline/2_1iterative_mask.py`
- [Fragment summaries](https://github.com/zixin22/fragfuse/tree/main/Agentharm/src/inspect_evals/agentharm/fragfuse_pipeline): `Agentharm/src/inspect_evals/agentharm/fragfuse_pipeline`
- [Host checks](https://github.com/zixin22/fragfuse/blob/main/Agentharm/src/inspect_evals/agentharm/fragfuse_pipeline/4host_check.py): `Agentharm/src/inspect_evals/agentharm/fragfuse_pipeline/4host_check.py`
- [Memory injection](https://github.com/zixin22/fragfuse/blob/main/Agentharm/src/inspect_evals/agentharm/fragfuse_pipeline/5injection_store.py): `Agentharm/src/inspect_evals/agentharm/fragfuse_pipeline/5injection_store.py`
- [Retrieval tests](https://github.com/zixin22/fragfuse/blob/main/Agentharm/src/inspect_evals/agentharm/fragfuse_pipeline/8retrieve_rate.py): `Agentharm/src/inspect_evals/agentharm/fragfuse_pipeline/8retrieve_rate.py`
- [Guardrail prompts](https://github.com/zixin22/fragfuse/blob/main/Agentharm/src/inspect_evals/agentharm/guardrail_prompt/guardrail_prompt.py): `Agentharm/src/inspect_evals/agentharm/guardrail_prompt/guardrail_prompt.py`
- [Evaluation scoring](https://github.com/zixin22/fragfuse/blob/main/Agentharm/src/inspect_evals/agentharm/scorer.py): `Agentharm/src/inspect_evals/agentharm/scorer.py`

### RAP WebShop

- [Attack runner](https://github.com/zixin22/fragfuse/blob/main/RAP/webshop/main.py): `RAP/webshop/main.py`
- [Fragment construction](https://github.com/zixin22/fragfuse/blob/main/RAP/webshop/fragExtracor_pipeline/frag_mask_pipeline.py): `RAP/webshop/fragExtracor_pipeline/frag_mask_pipeline.py`
- [WebShop prompts](https://github.com/zixin22/fragfuse/blob/main/RAP/webshop/prompts/webshop_prompt.py): `RAP/webshop/prompts/webshop_prompt.py`
- [Rule checking](https://github.com/zixin22/fragfuse/blob/main/RAP/webshop/rule_and_profile/rule_checker.py): `RAP/webshop/rule_and_profile/rule_checker.py`
- [User profiles](https://github.com/zixin22/fragfuse/blob/main/RAP/webshop/rule_and_profile/user_profile.py): `RAP/webshop/rule_and_profile/user_profile.py`
- [Rule examples](https://github.com/zixin22/fragfuse/blob/main/RAP/webshop/rule_and_profile/request_webshop.py): `RAP/webshop/rule_and_profile/request_webshop.py`
- [Rule metrics](https://github.com/zixin22/fragfuse/blob/main/RAP/webshop/rule_and_profile/metrics.py): `RAP/webshop/rule_and_profile/metrics.py`
- [WebShop guard adapter](https://github.com/zixin22/fragfuse/blob/main/RAP/guard%20agent/webshop_guard_agent.py): `RAP/guard agent/webshop_guard_agent.py`
- [Guard agent](https://github.com/zixin22/fragfuse/blob/main/RAP/guard%20agent/guardagent.py): `RAP/guard agent/guardagent.py`
- [Guard prompts](https://github.com/zixin22/fragfuse/blob/main/RAP/guard%20agent/prompts_guard.py): `RAP/guard agent/prompts_guard.py`
- [WebShop guard tools](https://github.com/zixin22/fragfuse/blob/main/RAP/guard%20agent/toolset_webshop.py): `RAP/guard agent/toolset_webshop.py`
- [Query optimization](https://github.com/zixin22/fragfuse/tree/main/RAP/webshop/optimization/optimization_webshop): `RAP/webshop/optimization/optimization_webshop`
- [Carrier templates](https://github.com/zixin22/fragfuse/blob/main/RAP/webshop/optimization/optimization_webshop/carrier_query_template_utils.py): `RAP/webshop/optimization/optimization_webshop/carrier_query_template_utils.py`

### OSAgent and SeeAct

- [OS experiment runner](https://github.com/zixin22/fragfuse/blob/main/OSAgent%20and%20SeeAct/AGrail4Agent/DAS/exp_OS.py): `OSAgent and SeeAct/AGrail4Agent/DAS/exp_OS.py`
- [Action guardrail](https://github.com/zixin22/fragfuse/blob/main/OSAgent%20and%20SeeAct/AGrail4Agent/DAS/guardrail.py): `OSAgent and SeeAct/AGrail4Agent/DAS/guardrail.py`
- [Guard prompts](https://github.com/zixin22/fragfuse/blob/main/OSAgent%20and%20SeeAct/AGrail4Agent/DAS/prompts.py): `OSAgent and SeeAct/AGrail4Agent/DAS/prompts.py`
- [Permission detection](https://github.com/zixin22/fragfuse/blob/main/OSAgent%20and%20SeeAct/AGrail4Agent/DAS/tools/permission_tool.py): `OSAgent and SeeAct/AGrail4Agent/DAS/tools/permission_tool.py`
- [Tool interface](https://github.com/zixin22/fragfuse/blob/main/OSAgent%20and%20SeeAct/AGrail4Agent/DAS/tools/tool.py): `OSAgent and SeeAct/AGrail4Agent/DAS/tools/tool.py`
- [Web detection](https://github.com/zixin22/fragfuse/blob/main/OSAgent%20and%20SeeAct/AGrail4Agent/DAS/tools/web_tool.py): `OSAgent and SeeAct/AGrail4Agent/DAS/tools/web_tool.py`
- [Code detection](https://github.com/zixin22/fragfuse/blob/main/OSAgent%20and%20SeeAct/AGrail4Agent/DAS/tools/code_tool.py): `OSAgent and SeeAct/AGrail4Agent/DAS/tools/code_tool.py`

### Other components

- **Researcher**
- **Benchmark operator**
- **Experiment operator**

## Connections

- Researcher → Benchmark task: configures (evidence: `Agentharm/src/inspect_evals/agentharm/agentharm.py`)
- Benchmark task → Agent variants: selects agent (evidence: `Agentharm/src/inspect_evals/agentharm/agentharm.py`)
- Memory agent → Agent variants: implements variant (evidence: `Agentharm/src/inspect_evals/agentharm/agents/agent.py`)
- Memory agent → Benchmark task: runs in task
- Benchmark task → Evaluation scoring: scores results (evidence: `Agentharm/src/inspect_evals/agentharm/agentharm.py`)
- Evaluation scoring → Benchmark task: uses metrics (evidence: `Agentharm/src/inspect_evals/agentharm/scorer.py`)
- Memory agent → Guardrail prompts: uses prompts (evidence: `Agentharm/src/inspect_evals/agentharm/agents/memory_agent.py`)
- Researcher → Fragment extraction: starts pipeline (evidence: `README.md`)
- Fragment extraction → Iterative masking: precedes masking
- Iterative masking → Fragment summaries: prepares fragments
- Fragment summaries → Host checks: feeds host checks
- Host checks → Memory injection: precedes injection
- Memory injection → Retrieval tests: followed by retrieval (evidence: `README.md`)
- Retrieval tests → Guardrail prompts: evaluates mitigation
- Benchmark operator → Attack runner: runs experiment (evidence: `README.md`)
- Attack runner → Fragment construction: uses attack inputs
- Attack runner → WebShop prompts: uses prompts (evidence: `RAP/webshop/main.py`)
- Attack runner → Rule checking: runs rule checks (evidence: `RAP/webshop/main.py`)
- Attack runner → WebShop guard adapter: selects defense (evidence: `RAP/webshop/main.py`)
- Attack runner → Rule metrics: evaluates outcomes
- Rule checking → User profiles: checks profile (evidence: `RAP/webshop/rule_and_profile/rule_checker.py`)
- Rule checking → Rule examples: uses rule examples
- WebShop guard adapter → Guard agent: delegates guard work (evidence: `RAP/guard agent/webshop_guard_agent.py`)
- WebShop guard adapter → WebShop guard tools: uses toolset (evidence: `RAP/guard agent/webshop_guard_agent.py`)
- WebShop guard adapter → User profiles: uses profile data (evidence: `RAP/guard agent/webshop_guard_agent.py`)
- WebShop guard adapter → Rule examples: uses rule examples (evidence: `RAP/guard agent/webshop_guard_agent.py`)
- Guard agent → Guard prompts: loads prompts (evidence: `RAP/guard agent/guardagent.py`)
- WebShop guard tools → Guard prompts: uses prompt definitions (evidence: `RAP/guard agent/toolset_webshop.py`)
- Benchmark operator → Query optimization: optimizes queries
- Query optimization → Carrier templates: rewrites templates (evidence: `RAP/webshop/optimization/optimization_webshop/evaluator.py`)
- Fragment construction → Carrier templates: builds carrier queries
- Experiment operator → OS experiment runner: runs experiment (evidence: `README.md`)
- OS experiment runner → Guard prompts: uses prompts (evidence: `OSAgent and SeeAct/AGrail4Agent/DAS/exp_OS.py`)
- OS experiment runner → Action guardrail: invokes guardrail (evidence: `OSAgent and SeeAct/AGrail4Agent/DAS/exp_OS.py`)
- Action guardrail → Permission detection: dispatches checks (evidence: `OSAgent and SeeAct/AGrail4Agent/DAS/guardrail.py`)
- Action guardrail → Web detection: dispatches checks (evidence: `OSAgent and SeeAct/AGrail4Agent/DAS/guardrail.py`)
- Action guardrail → Code detection: dispatches checks (evidence: `OSAgent and SeeAct/AGrail4Agent/DAS/guardrail.py`)
- Permission detection → Tool interface: inherits interface (evidence: `OSAgent and SeeAct/AGrail4Agent/DAS/tools/permission_tool.py`)
- Web detection → Tool interface: inherits interface (evidence: `OSAgent and SeeAct/AGrail4Agent/DAS/tools/web_tool.py`)
- Code detection → Tool interface: inherits interface (evidence: `OSAgent and SeeAct/AGrail4Agent/DAS/tools/code_tool.py`)
