# sudarshan2412/taskverifier architecture

> Architecture diagram of the GitHub repository sudarshan2412/taskverifier, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-09-28). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/sudarshan2412/taskverifier
- Repository: https://github.com/sudarshan2412/taskverifier

## Overview

TaskVerifier generates C proof-of-concept exploits for vulnerability tasks, then iteratively compiles and runs them, returning structured feedback to the LLM until a crash is reproduced or attempts are exhausted. Its main path is dataset selection and sanitization → agent generation/refinement → verifier checks → result and pass/fail evaluation. The README also documents one-shot baseline runs and batch comparison/metrics. Some subsystem links reflect README documentation because those source files were not sampled.

Main entry points: [`agent/agent_loop.py`](https://github.com/sudarshan2412/taskverifier/blob/main/agent/agent_loop.py) (Retry loop), [`verifier/__init__.py`](https://github.com/sudarshan2412/taskverifier/blob/main/verifier/__init__.py) (Verifier pipeline), [`run_pipeline.py`](https://github.com/sudarshan2412/taskverifier/blob/main/run_pipeline.py) (Trial pipeline), [`results.json`](https://github.com/sudarshan2412/taskverifier/blob/main/results.json) (Trial results)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_orchestration["Task orchestration"]
  node_dataset[("CVE subset")]
  node_pipeline["Trial pipeline<br/>[run_pipeline.py]"]
  node_sanitizer["Entry sanitizer"]
end

subgraph group_generation["PoC generation"]
  node_agent["Retry loop<br/>[agent_loop.py]"]
  node_prompts["Prompt builder<br/>[prompt_builder.py]"]
  node_context["Conversation context<br/>[context_manager.py]"]
  node_llm["LLM client<br/>[llm_client.py]"]
  node_extractor["Code extractor<br/>[code_extractor.py]"]
  node_hints["Format hints<br/>[format_hints.py]"]
end

subgraph group_verification["Verification"]
  node_verifier["Verifier pipeline<br/>[__init__.py]"]
  node_compiler["Compiler checks<br/>[compiler.py]"]
  node_execution["Sandbox execution<br/>[execution.py]"]
  node_sanitizer_parse["Crash parsing<br/>[sanitizer.py]"]
  node_feedback["Feedback builder"]
  node_hallucination["Symbol detection"]
end

subgraph group_benchmark["Benchmark and evaluation"]
  node_evaluator["Pass/fail evaluator<br/>[evaluator.py]"]
  node_baseline["Baseline runner<br/>[baseline_runner.py]"]
  node_runner["Trial runner<br/>[runner.py]"]
  node_metrics["Metrics analysis<br/>[compute_metrics.py]"]
  node_plots["Result plots<br/>[plot_results.py]"]
  node_results[("Trial results<br/>[results.json]")]
end

node_researcher(("Researcher"))
node_openrouter["OpenRouter API"]
node_docker["Vulnerable target"]

node_researcher -->|"starts trials"| node_pipeline
node_dataset -->|"supplies tasks"| node_pipeline
node_pipeline -->|"sanitizes entries"| node_sanitizer
node_pipeline -->|"runs trials"| node_agent
node_agent -->|"builds prompts"| node_prompts
node_agent -->|"manages history"| node_context
node_agent -->|"requests generation"| node_llm
node_llm -->|"sends requests"| node_openrouter
node_agent -->|"extracts code"| node_extractor
node_prompts -->|"uses format hints"| node_hints
node_agent -->|"checks symbols"| node_hallucination
node_agent -->|"submits PoC"| node_verifier
node_verifier -->|"compiles PoC"| node_compiler
node_verifier -->|"runs compiled PoC"| node_execution
node_execution -->|"runs in sandbox"| node_docker
node_verifier -->|"parses crash output"| node_sanitizer_parse
node_verifier -->|"builds guidance"| node_feedback
node_verifier -->|"returns feedback"| node_agent
node_pipeline -->|"evaluates outcome"| node_evaluator
node_baseline -->|"runs baseline trials"| node_runner
node_runner -->|"invokes agent"| node_agent
node_runner -->|"records outcomes"| node_results
node_pipeline -->|"writes outcomes"| node_results
node_results -->|"aggregates results"| node_metrics
node_results -->|"visualizes results"| node_plots

click node_dataset "https://github.com/sudarshan2412/taskverifier/blob/main/cybergym_subset.json"
click node_pipeline "https://github.com/sudarshan2412/taskverifier/blob/main/run_pipeline.py"
click node_sanitizer "https://github.com/sudarshan2412/taskverifier/blob/main/dataset_sanitizer.py"
click node_agent "https://github.com/sudarshan2412/taskverifier/blob/main/agent/agent_loop.py"
click node_prompts "https://github.com/sudarshan2412/taskverifier/blob/main/agent/prompt_builder.py"
click node_context "https://github.com/sudarshan2412/taskverifier/blob/main/agent/context_manager.py"
click node_llm "https://github.com/sudarshan2412/taskverifier/blob/main/agent/llm_client.py"
click node_extractor "https://github.com/sudarshan2412/taskverifier/blob/main/agent/code_extractor.py"
click node_hints "https://github.com/sudarshan2412/taskverifier/blob/main/agent/format_hints.py"
click node_verifier "https://github.com/sudarshan2412/taskverifier/blob/main/verifier/__init__.py"
click node_compiler "https://github.com/sudarshan2412/taskverifier/blob/main/verifier/compiler.py"
click node_execution "https://github.com/sudarshan2412/taskverifier/blob/main/verifier/execution.py"
click node_sanitizer_parse "https://github.com/sudarshan2412/taskverifier/blob/main/verifier/sanitizer.py"
click node_feedback "https://github.com/sudarshan2412/taskverifier/blob/main/verifier/feedback_builder.py"
click node_hallucination "https://github.com/sudarshan2412/taskverifier/blob/main/verifier/hallucination_detector.py"
click node_evaluator "https://github.com/sudarshan2412/taskverifier/blob/main/evaluator.py"
click node_baseline "https://github.com/sudarshan2412/taskverifier/blob/main/baseline_runner.py"
click node_runner "https://github.com/sudarshan2412/taskverifier/blob/main/runner.py"
click node_metrics "https://github.com/sudarshan2412/taskverifier/blob/main/compute_metrics.py"
click node_plots "https://github.com/sudarshan2412/taskverifier/blob/main/plot_results.py"
click node_results "https://github.com/sudarshan2412/taskverifier/blob/main/results.json"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_dataset,node_pipeline,node_sanitizer toneBlue
class node_agent,node_prompts,node_context,node_llm,node_extractor,node_hints toneAmber
class node_verifier,node_compiler,node_execution,node_sanitizer_parse,node_feedback,node_hallucination,node_openrouter toneMint
class node_evaluator,node_baseline,node_runner,node_metrics,node_plots,node_results toneRose
class node_researcher,node_docker toneIndigo
```

## Components

### Task orchestration

- [CVE subset](https://github.com/sudarshan2412/taskverifier/blob/main/cybergym_subset.json): `cybergym_subset.json`
- [Trial pipeline](https://github.com/sudarshan2412/taskverifier/blob/main/run_pipeline.py): `run_pipeline.py`
- [Entry sanitizer](https://github.com/sudarshan2412/taskverifier/blob/main/dataset_sanitizer.py): `dataset_sanitizer.py`

### PoC generation

- [Retry loop](https://github.com/sudarshan2412/taskverifier/blob/main/agent/agent_loop.py): `agent/agent_loop.py`
- [Prompt builder](https://github.com/sudarshan2412/taskverifier/blob/main/agent/prompt_builder.py): `agent/prompt_builder.py`
- [Conversation context](https://github.com/sudarshan2412/taskverifier/blob/main/agent/context_manager.py): `agent/context_manager.py`
- [LLM client](https://github.com/sudarshan2412/taskverifier/blob/main/agent/llm_client.py): `agent/llm_client.py`
- [Code extractor](https://github.com/sudarshan2412/taskverifier/blob/main/agent/code_extractor.py): `agent/code_extractor.py`
- [Format hints](https://github.com/sudarshan2412/taskverifier/blob/main/agent/format_hints.py): `agent/format_hints.py`

### Verification

- [Verifier pipeline](https://github.com/sudarshan2412/taskverifier/blob/main/verifier/__init__.py): `verifier/__init__.py`
- [Compiler checks](https://github.com/sudarshan2412/taskverifier/blob/main/verifier/compiler.py): `verifier/compiler.py`
- [Sandbox execution](https://github.com/sudarshan2412/taskverifier/blob/main/verifier/execution.py): `verifier/execution.py`
- [Crash parsing](https://github.com/sudarshan2412/taskverifier/blob/main/verifier/sanitizer.py): `verifier/sanitizer.py`
- [Feedback builder](https://github.com/sudarshan2412/taskverifier/blob/main/verifier/feedback_builder.py): `verifier/feedback_builder.py`
- [Symbol detection](https://github.com/sudarshan2412/taskverifier/blob/main/verifier/hallucination_detector.py): `verifier/hallucination_detector.py`

### Benchmark and evaluation

- [Pass/fail evaluator](https://github.com/sudarshan2412/taskverifier/blob/main/evaluator.py): `evaluator.py`
- [Baseline runner](https://github.com/sudarshan2412/taskverifier/blob/main/baseline_runner.py): `baseline_runner.py`
- [Trial runner](https://github.com/sudarshan2412/taskverifier/blob/main/runner.py): `runner.py`
- [Metrics analysis](https://github.com/sudarshan2412/taskverifier/blob/main/compute_metrics.py): `compute_metrics.py`
- [Result plots](https://github.com/sudarshan2412/taskverifier/blob/main/plot_results.py): `plot_results.py`
- [Trial results](https://github.com/sudarshan2412/taskverifier/blob/main/results.json): `results.json`

### Other components

- **Researcher**
- **OpenRouter API**
- **Vulnerable target**

## Connections

- Researcher → Trial pipeline: starts trials
- CVE subset → Trial pipeline: supplies tasks
- Trial pipeline → Entry sanitizer: sanitizes entries
- Trial pipeline → Retry loop: runs trials
- Retry loop → Prompt builder: builds prompts
- Retry loop → Conversation context: manages history
- Retry loop → LLM client: requests generation
- LLM client → OpenRouter API: sends requests
- Retry loop → Code extractor: extracts code
- Prompt builder → Format hints: uses format hints
- Retry loop → Symbol detection: checks symbols
- Retry loop → Verifier pipeline: submits PoC
- Verifier pipeline → Compiler checks: compiles PoC
- Verifier pipeline → Sandbox execution: runs compiled PoC
- Sandbox execution → Vulnerable target: runs in sandbox
- Verifier pipeline → Crash parsing: parses crash output
- Verifier pipeline → Feedback builder: builds guidance
- Verifier pipeline → Retry loop: returns feedback
- Trial pipeline → Pass/fail evaluator: evaluates outcome
- Baseline runner → Trial runner: runs baseline trials
- Trial runner → Retry loop: invokes agent
- Trial runner → Trial results: records outcomes
- Trial pipeline → Trial results: writes outcomes
- Trial results → Metrics analysis: aggregates results
- Trial results → Result plots: visualizes results