# sharathbaddam/agentwhetters-cybergym architecture

> Architecture diagram of the GitHub repository sharathbaddam/agentwhetters-cybergym, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-06-18). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/sharathbaddam/agentwhetters-cybergym
- Repository: https://github.com/sharathbaddam/agentwhetters-cybergym

## Overview

1. **What this repo is**
   - This is a Python-based “purple agent” for CyberGym/AgentBeats that generates exploit PoCs against a vulnerable program provided by a separate green-agent benchmark harness.
   - The main runtime is a long-lived agent service; the repo also includes a client/runner package for driving benchmark scenarios.

2. **Top-level split: service side vs. client side**
   - `src/` contains the production agent server and reasoning pipeline.
   - `agentbeats/` contains client-facing helpers, CLI, and models for interacting with the AgentBeats/CyberGym task flow.
   - `scenario*.toml` files and `run_scenario.py` indicate scripted benchmark runs rather than a general-purpose application UI.

3. **Inbound task boundary**
   - The green agent sends a task bundle to the server over an A2A-style interface: task text plus attachments such as `description.txt`, `error.txt`, and a vulnerable source tarball.
   - `src/server.py` exposes the server on port 9122.
   - `src/executor.py` keeps one `Agent` instance per `context_id`, so each benchmark task has isolated state and concurrent tasks do not interfere.

4. **Core control loop**
   - `src/agent.py` is the orchestration center.
   - It runs the entire exploit-generation loop: pre-analysis, LLM prompting, tool execution, PoC submission, feedback handling, retry, and final success reporting.
   - The agent caps work with both iteration and token budgets, so the architecture is explicitly cost-governed rather than open-ended.

5. **Zero-token pre-analysis pipeline**
   - Before any model call, the agent runs structured analysis modules that do not consume LLM tokens:
     - `hypothesis_parser.py` extracts a normalized vulnerability signal from the task description and error text.
     - `codebase_triage.py` ranks source files in the vulnerable repo by likely relevance to the detected vulnerability class.
     - `taint_extractor.py` traces likely input-to-crash paths and identifies trigger fields, magic bytes, and format structure.
     - `memory_system.py` retrieves similar past tasks and known failed strategies.
   - This is the main architectural optimization: the LLM is only called after the search space has been narrowed.

6. **LLM interaction boundary**
   - The agent supports two OpenAI API modes: Responses API for newer reasoning models and Chat Completions for classic models.
   - The LLM is not asked to discover everything from scratch; it receives parsed vulnerability metadata, ranked files, taint/path hints, and warm-start context.
   - The model can invoke two tools:
     - `execute_python` to synthesize or transform PoC bytes locally.
     - `submit_poc` to send a candidate artifact back through the benchmark round-trip for testing.

7. **Feedback loop and repair**
   - After the green agent tests a PoC, `feedback_classifier.py` reduces raw exits/ASan output into a small set of categories such as `NO_CRASH`, `WRONG_LOCATION`, `PARTIAL_CRASH`, or `PARSER_REJECTED`.
   - `binary_mutator.py` then handles near-miss cases by producing deterministic byte-level mutations without another LLM call.
   - Only when the classifier says the issue needs reasoning-level adjustment does the agent spend another model iteration.

8. **Cross-task learning and persistence**
   - `memory_system.py` stores successful exploit metadata and prior failed strategies in JSON-backed memory, so later tasks can reuse patterns from similar vulnerability classes or domains.
   - `token_tracker.py` records model usage, latency, and estimated cost and enforces per-task budgets.
   - These are persistence boundaries: the exploit loop is ephemeral, but memory and token accounting survive across tasks for analysis and reuse.

9. **Support code and observability**
   - `log_config.py` defines a custom `TRACE` level for deep debugging of the agent’s internal decisions.
   - `src/messenger.py` and `src/run_scenario.py` support task messaging and scenario execution around the main agent flow.
   - The repo includes a Dockerfile for containerized execution and `.github/workflows/test-and-publish.yml` for CI/release automation.

10. **Testing and validation**
   - `tests/` covers parser behavior, feedback classification, agent behavior, and integration paths.
   - The test surface reflects the architecture: most correctness risk is in structured extraction, feedback interpretation, and end-to-end task handling rather than in a traditional web backend.

11. **Important external dependencies and runtime assumptions**
   - The agent depends on Python, `uv`, and either OpenAI or Azure OpenAI for the reasoning loop.
   - It also assumes CyberGym’s green-agent test harness and A2A-style task exchange.
   - `taint_extractor.py` can use `cscope` when present, but falls back to grep-style analysis, so source-code tracing is designed to degrade gracefully.

12. **How to diagram it**
   - The clearest architecture diagram is: green-agent task ingress → per-context executor → agent orchestration → zero-token analysis modules + memory retrieval → LLM/tool loop → green-agent test round-trip → classifier/mutator repair loop → success artifact and memory persistence.
   - The main system boundary is the agent service; the benchmark harness and client package are external consumers, not part of the core exploit reasoning pipeline.

Main entry points: [`src/agent.py`](https://github.com/sharathbaddam/agentwhetters-cybergym/blob/main/src/agent.py) (Agent loop), [`agentbeats/client.py`](https://github.com/sharathbaddam/agentwhetters-cybergym/blob/main/agentbeats/client.py) (Client), [`src/server.py`](https://github.com/sharathbaddam/agentwhetters-cybergym/blob/main/src/server.py) (Server), [`src/executor.py`](https://github.com/sharathbaddam/agentwhetters-cybergym/blob/main/src/executor.py) (Executor)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_service["Agent service"]
  node_server["Server<br/>A2A service<br/>[server.py]"]
  node_executor["Executor<br/>context router<br/>[executor.py]"]
  node_agent["Agent loop<br/>orchestrator<br/>[agent.py]"]
  node_hypothesis_parser["Hypothesis parser<br/>task parser"]
  node_codebase_triage["Code triage<br/>file ranker<br/>[codebase_triage.py]"]
  node_taint_extractor["Taint extractor<br/>path analyzer<br/>[taint_extractor.py]"]
  node_feedback_classifier["Feedback classifier<br/>result classifier"]
  node_binary_mutator["Mutator<br/>[binary_mutator.py]"]
  node_messenger["Messenger<br/>task messaging<br/>[messenger.py]"]
end

subgraph group_client["Client side"]
  node_client["Client<br/>[client.py]"]
  node_cli["CLI<br/>command line<br/>[client_cli.py]"]
  node_scenario_runner["Scenario runner<br/>scenario driver<br/>[run_scenario.py]"]
  node_models["Models<br/>[models.py]"]
end

subgraph group_persistence["State"]
  node_memory_system[("Memory<br/>[memory_system.py]")]
  node_token_tracker["Token tracker<br/>budgeting<br/>[token_tracker.py]"]
  node_memory_store[("Memory store<br/>json data<br/>[solved_tasks.json]")]
  node_token_store[("Usage store<br/>json data<br/>[token_usage.json]")]
end

subgraph group_ops["Ops"]
  node_dockerfile["Dockerfile<br/>container build"]
  node_workflow["CI workflow<br/>github actions"]
end

node_green_harness(("Green harness<br/>external benchmark"))
node_openai_api(("LLM API<br/>external reasoning"))

node_green_harness -->|"task bundle"| node_server
node_server -->|"dispatch"| node_executor
node_executor -->|"context agent"| node_agent
node_agent -->|"pre-analyze"| node_hypothesis_parser
node_agent -->|"pre-analyze"| node_codebase_triage
node_agent -->|"pre-analyze"| node_taint_extractor
node_agent -->|"retrieve"| node_memory_system
node_memory_system -->|"persist"| node_memory_store
node_agent -->|"budget"| node_token_tracker
node_token_tracker -->|"persist"| node_token_store
node_agent -->|"reason"| node_openai_api
node_agent -->|"analyze result"| node_feedback_classifier
node_feedback_classifier -->|"near miss"| node_binary_mutator
node_binary_mutator -->|"retry"| node_agent
node_agent -->|"submit"| node_messenger
node_cli -->|"invoke"| node_client
node_scenario_runner -->|"run"| node_client
node_client -->|"drive tasks"| node_green_harness
node_client -->|"agent task"| node_server
node_models -.->|"types"| node_client
node_dockerfile -.->|"package"| node_server
node_workflow -.->|"build test"| node_dockerfile

click node_server "https://github.com/sharathbaddam/agentwhetters-cybergym/blob/main/src/server.py"
click node_executor "https://github.com/sharathbaddam/agentwhetters-cybergym/blob/main/src/executor.py"
click node_agent "https://github.com/sharathbaddam/agentwhetters-cybergym/blob/main/src/agent.py"
click node_hypothesis_parser "https://github.com/sharathbaddam/agentwhetters-cybergym/blob/main/src/hypothesis_parser.py"
click node_codebase_triage "https://github.com/sharathbaddam/agentwhetters-cybergym/blob/main/src/codebase_triage.py"
click node_taint_extractor "https://github.com/sharathbaddam/agentwhetters-cybergym/blob/main/src/taint_extractor.py"
click node_memory_system "https://github.com/sharathbaddam/agentwhetters-cybergym/blob/main/src/memory_system.py"
click node_token_tracker "https://github.com/sharathbaddam/agentwhetters-cybergym/blob/main/src/token_tracker.py"
click node_feedback_classifier "https://github.com/sharathbaddam/agentwhetters-cybergym/blob/main/src/feedback_classifier.py"
click node_binary_mutator "https://github.com/sharathbaddam/agentwhetters-cybergym/blob/main/src/binary_mutator.py"
click node_messenger "https://github.com/sharathbaddam/agentwhetters-cybergym/blob/main/src/messenger.py"
click node_client "https://github.com/sharathbaddam/agentwhetters-cybergym/blob/main/agentbeats/client.py"
click node_cli "https://github.com/sharathbaddam/agentwhetters-cybergym/blob/main/agentbeats/client_cli.py"
click node_scenario_runner "https://github.com/sharathbaddam/agentwhetters-cybergym/blob/main/agentbeats/run_scenario.py"
click node_models "https://github.com/sharathbaddam/agentwhetters-cybergym/blob/main/agentbeats/models.py"
click node_memory_store "https://github.com/sharathbaddam/agentwhetters-cybergym/blob/main/src/memory/solved_tasks.json"
click node_token_store "https://github.com/sharathbaddam/agentwhetters-cybergym/blob/main/src/memory/token_usage.json"
click node_dockerfile "https://github.com/sharathbaddam/agentwhetters-cybergym/tree/main/Dockerfile"
click node_workflow "https://github.com/sharathbaddam/agentwhetters-cybergym/blob/main/.github/workflows/test-and-publish.yml"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_server,node_executor,node_agent,node_hypothesis_parser,node_codebase_triage,node_taint_extractor,node_feedback_classifier,node_binary_mutator,node_messenger toneBlue
class node_client,node_cli,node_scenario_runner,node_models toneAmber
class node_memory_system,node_token_tracker,node_memory_store,node_token_store toneMint
class node_dockerfile,node_workflow toneRose
class node_green_harness,node_openai_api toneNeutral
```

## Components

### Agent service

Core purple-agent runtime and reasoning pipeline

- [Server](https://github.com/sharathbaddam/agentwhetters-cybergym/blob/main/src/server.py): A2A service · Inbound task service exposed on port 9122 · `src/server.py`
- [Executor](https://github.com/sharathbaddam/agentwhetters-cybergym/blob/main/src/executor.py): context router · Maintains one agent instance per context_id · `src/executor.py`
- [Agent loop](https://github.com/sharathbaddam/agentwhetters-cybergym/blob/main/src/agent.py): orchestrator · Main exploit-generation control loop with budgets and retries · `src/agent.py`
- [Hypothesis parser](https://github.com/sharathbaddam/agentwhetters-cybergym/blob/main/src/hypothesis_parser.py): task parser · Normalizes vulnerability signals from task text and errors · `src/hypothesis_parser.py`
- [Code triage](https://github.com/sharathbaddam/agentwhetters-cybergym/blob/main/src/codebase_triage.py): file ranker · Ranks source files by likely relevance · `src/codebase_triage.py`
- [Taint extractor](https://github.com/sharathbaddam/agentwhetters-cybergym/blob/main/src/taint_extractor.py): path analyzer · Finds likely input-to-crash paths and trigger structure · `src/taint_extractor.py`
- [Feedback classifier](https://github.com/sharathbaddam/agentwhetters-cybergym/blob/main/src/feedback_classifier.py): result classifier · Maps test output to coarse feedback categories · `src/feedback_classifier.py`
- [Mutator](https://github.com/sharathbaddam/agentwhetters-cybergym/blob/main/src/binary_mutator.py): byte mutator · Performs deterministic near-miss PoC mutations · `src/binary_mutator.py`
- [Messenger](https://github.com/sharathbaddam/agentwhetters-cybergym/blob/main/src/messenger.py): task messaging · Supports task exchange and scenario messaging · `src/messenger.py`

### Client side

Benchmark-driving helpers and scenario entrypoints

- [Client](https://github.com/sharathbaddam/agentwhetters-cybergym/blob/main/agentbeats/client.py): agentbeats client · Client-facing task runner and benchmark helper · `agentbeats/client.py`
- [CLI](https://github.com/sharathbaddam/agentwhetters-cybergym/blob/main/agentbeats/client_cli.py): command line · Command-line entrypoint for running tasks · `agentbeats/client_cli.py`
- [Scenario runner](https://github.com/sharathbaddam/agentwhetters-cybergym/blob/main/agentbeats/run_scenario.py): scenario driver · Runs scripted benchmark scenarios · `agentbeats/run_scenario.py`
- [Models](https://github.com/sharathbaddam/agentwhetters-cybergym/blob/main/agentbeats/models.py): task models · Shared data models for AgentBeats flow · `agentbeats/models.py`

### State

Cross-task memory and accounting data

- [Memory](https://github.com/sharathbaddam/agentwhetters-cybergym/blob/main/src/memory_system.py): task memory · Retrieves and stores similar tasks and strategies · `src/memory_system.py`
- [Token tracker](https://github.com/sharathbaddam/agentwhetters-cybergym/blob/main/src/token_tracker.py): budgeting · Tracks usage, latency, and per-task limits · `src/token_tracker.py`
- [Memory store](https://github.com/sharathbaddam/agentwhetters-cybergym/blob/main/src/memory/solved_tasks.json): json data · Persisted solved tasks and token usage records · `src/memory/solved_tasks.json`
- [Usage store](https://github.com/sharathbaddam/agentwhetters-cybergym/blob/main/src/memory/token_usage.json): json data · Persisted token usage history · `src/memory/token_usage.json`

### Ops

Containerization and release automation

- [Dockerfile](https://github.com/sharathbaddam/agentwhetters-cybergym/tree/main/Dockerfile): container build · Containerized execution environment · `Dockerfile`
- [CI workflow](https://github.com/sharathbaddam/agentwhetters-cybergym/blob/main/.github/workflows/test-and-publish.yml): github actions · Test and publish automation · `.github/workflows/test-and-publish.yml`

### Other components

- **Green harness**: external benchmark · CyberGym/AgentBeats task source and PoC tester
- **LLM API**: external reasoning · OpenAI or Azure OpenAI backend used by the agent

## Connections

- Green harness → Server: task bundle · Green-agent task ingress with attachments
- Server → Executor: dispatch · Routes incoming tasks by context_id
- Executor → Agent loop: context agent · Creates or reuses one agent per task context
- Agent loop → Hypothesis parser: pre-analyze · Extracts normalized vulnerability signals
- Agent loop → Code triage: pre-analyze · Ranks relevant source files
- Agent loop → Taint extractor: pre-analyze · Derives trigger paths and field hints
- Agent loop → Memory: retrieve · Loads similar tasks and prior failures
- Memory → Memory store: persist · Reads and writes JSON-backed task memory
- Agent loop → Token tracker: budget · Enforces token and latency limits
- Token tracker → Usage store: persist · Stores usage accounting
- Agent loop → LLM API: reason · Calls Responses API or Chat Completions
- Agent loop → Feedback classifier: analyze result · Reduces test output to coarse categories
- Feedback classifier → Mutator: near miss · Triggers deterministic repair mutations
- Mutator → Agent loop: retry · Feeds improved candidate back into the loop
- Agent loop → Messenger: submit · Sends PoCs and receives benchmark feedback
- CLI → Client: invoke · CLI drives client helpers
- Scenario runner → Client: run · Scenario execution uses the client package
- Client → Green harness: drive tasks · Client participates in benchmark task flow
- Client → Server: agent task · Submits or coordinates task exchange with the service
- Models → Client: types · Shared request and response structures
- Dockerfile → Server: package · Containerizes the runtime
- CI workflow → Dockerfile: build test · CI validates and publishes containerized release