# tencent/workbuddy-bench architecture

> Architecture diagram of the GitHub repository tencent/workbuddy-bench, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-11-17). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/tencent/workbuddy-bench
- Repository: https://github.com/tencent/workbuddy-bench

## Overview

WorkBuddy Bench is an evaluation framework for coding agents across Code, Web, Office, and Security tasks. Its principal workflow configures a model, harness, and dataset; prepares and runs task trials in Docker; optionally routes model traffic through a logging/protocol proxy; then verifies results and scores the evaluation. The graph includes the documented evaluation stages and major verifier and proxy capabilities. Some runner and post-processing internals were not excerpted, so links not supported by the README or source index are omitted.

Main entry points: [`src/workbuddy_bench/runner/sharded_eval.py`](https://github.com/tencent/workbuddy-bench/blob/main/src/workbuddy_bench/runner/sharded_eval.py) (Trial orchestration), [`src/workbuddy_bench/proxy/pipeline.py`](https://github.com/tencent/workbuddy-bench/blob/main/src/workbuddy_bench/proxy/pipeline.py) (Protocol and interceptors), [`src/workbuddy_bench/runner/resolve_manifest.py`](https://github.com/tencent/workbuddy-bench/blob/main/src/workbuddy_bench/runner/resolve_manifest.py) (Run manifest), [`src/workbuddy_bench/judge/core/engine.py`](https://github.com/tencent/workbuddy-bench/blob/main/src/workbuddy_bench/judge/core/engine.py) (Composite verifier engine)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_setup["Evaluation setup"]
  node_job_config["Job and model config<br/>[config_loaders.py]"]
  node_manifest["Run manifest"]
  node_task_prep["Task preparation<br/>[prepare_tasks.py]"]
  node_subsets["Benchmark subsets"]
end

subgraph group_execution["Trial execution"]
  node_trial_runner["Trial orchestration<br/>[sharded_eval.py]"]
  node_docker["Docker sandbox"]
  node_harness["Agent harness adapters"]
  node_agent_usage["Agent usage capture<br/>[_agent_user.py]"]
end

subgraph group_proxy["Model traffic proxy"]
  node_proxy_config["Proxy routes<br/>[config.py]"]
  node_proxy_api["Proxy request entry<br/>[main.py]"]
  node_proxy_pipeline["Protocol and interceptors<br/>[pipeline.py]"]
  node_protocol_map["Anthropic/OpenAI mapping<br/>[a2o.py]"]
  node_request_logging["Request logging<br/>[logger.py]"]
end

subgraph group_evaluation["Verification and scoring"]
  node_profile_dispatch["Judge profile dispatch<br/>[dispatcher.py]"]
  node_verifier_engine["Composite verifier engine<br/>[engine.py]"]
  node_pytest_judge["Pytest and rule judges<br/>[harbor_pytest.py]"]
  node_artifact_evidence["Artifact inspection"]
  node_command_runtime["Command execution<br/>[command.py]"]
  node_score["Correctness scoring<br/>[scorer.py]"]
  node_llm_judge["LLM post-judge<br/>[llm_judge.py]"]
  node_results["Evaluation results"]
end

node_operator(("Benchmark operator"))
node_agent_cli(("Agent CLI"))
node_model_backend(("Model backend"))

node_operator -->|"configures"| node_job_config
node_job_config -->|"resolves"| node_manifest
node_manifest -.->|"prepares tasks"| node_task_prep
node_subsets -->|"supplies tasks"| node_task_prep
node_manifest -.->|"launches trials"| node_trial_runner
node_trial_runner -->|"runs in"| node_docker
node_trial_runner -->|"dispatches"| node_harness
node_harness -->|"runs"| node_agent_cli
node_agent_cli -.->|"sends model requests"| node_proxy_api
node_agent_usage -.->|"captures usage"| node_trial_runner
node_manifest -.->|"configures routes"| node_proxy_config
node_proxy_api -->|"dispatches requests"| node_proxy_pipeline
node_proxy_pipeline -->|"uses routes"| node_proxy_config
node_proxy_pipeline -->|"converts protocol"| node_protocol_map
node_proxy_pipeline -->|"logs exchanges"| node_request_logging
node_proxy_pipeline -->|"forwards requests"| node_model_backend
node_trial_runner -.->|"routes judging"| node_profile_dispatch
node_profile_dispatch -.->|"dispatches profile"| node_verifier_engine
node_verifier_engine -.->|"runs judges"| node_pytest_judge
node_pytest_judge -->|"executes commands"| node_command_runtime
node_verifier_engine -.->|"collects evidence"| node_artifact_evidence
node_trial_runner -->|"scores trial"| node_score
node_score -->|"writes scores"| node_results
node_results -->|"post-processes"| node_llm_judge
node_llm_judge -->|"updates scores"| node_results

click node_job_config "https://github.com/tencent/workbuddy-bench/blob/main/src/workbuddy_bench/runner/config_loaders.py"
click node_manifest "https://github.com/tencent/workbuddy-bench/blob/main/src/workbuddy_bench/runner/resolve_manifest.py"
click node_task_prep "https://github.com/tencent/workbuddy-bench/blob/main/src/workbuddy_bench/runner/prepare_tasks.py"
click node_trial_runner "https://github.com/tencent/workbuddy-bench/blob/main/src/workbuddy_bench/runner/sharded_eval.py"
click node_docker "https://github.com/tencent/workbuddy-bench/blob/main/src/workbuddy_bench/runner/docker_environment.py"
click node_harness "https://github.com/tencent/workbuddy-bench/blob/main/src/workbuddy_bench/runner/harness_adapters.py"
click node_agent_usage "https://github.com/tencent/workbuddy-bench/blob/main/src/workbuddy_bench/agents/_agent_user.py"
click node_proxy_config "https://github.com/tencent/workbuddy-bench/blob/main/src/workbuddy_bench/proxy/config.py"
click node_proxy_api "https://github.com/tencent/workbuddy-bench/blob/main/src/workbuddy_bench/proxy/main.py"
click node_proxy_pipeline "https://github.com/tencent/workbuddy-bench/blob/main/src/workbuddy_bench/proxy/pipeline.py"
click node_protocol_map "https://github.com/tencent/workbuddy-bench/blob/main/src/workbuddy_bench/proxy/protocols/a2o.py"
click node_request_logging "https://github.com/tencent/workbuddy-bench/blob/main/src/workbuddy_bench/proxy/interceptors/logger.py"
click node_profile_dispatch "https://github.com/tencent/workbuddy-bench/blob/main/src/workbuddy_bench/judge/profiles/dispatcher.py"
click node_verifier_engine "https://github.com/tencent/workbuddy-bench/blob/main/src/workbuddy_bench/judge/core/engine.py"
click node_pytest_judge "https://github.com/tencent/workbuddy-bench/blob/main/src/workbuddy_bench/judge/runners/harbor_pytest.py"
click node_artifact_evidence "https://github.com/tencent/workbuddy-bench/blob/main/src/workbuddy_bench/judge/evidence/artifact_inspector.py"
click node_command_runtime "https://github.com/tencent/workbuddy-bench/blob/main/src/workbuddy_bench/judge/runtime/command.py"
click node_score "https://github.com/tencent/workbuddy-bench/blob/main/src/workbuddy_bench/scorer/scorer.py"
click node_llm_judge "https://github.com/tencent/workbuddy-bench/blob/main/src/workbuddy_bench/scorer/llm_judge.py"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_job_config,node_manifest,node_task_prep,node_subsets toneBlue
class node_trial_runner,node_docker,node_harness,node_agent_usage toneAmber
class node_proxy_config,node_proxy_api,node_proxy_pipeline,node_protocol_map,node_request_logging toneMint
class node_profile_dispatch,node_verifier_engine,node_pytest_judge,node_artifact_evidence,node_command_runtime,node_score,node_llm_judge,node_results toneRose
class node_operator,node_agent_cli,node_model_backend toneIndigo
```

## Components

### Evaluation setup

- [Job and model config](https://github.com/tencent/workbuddy-bench/blob/main/src/workbuddy_bench/runner/config_loaders.py): `src/workbuddy_bench/runner/config_loaders.py`
- [Run manifest](https://github.com/tencent/workbuddy-bench/blob/main/src/workbuddy_bench/runner/resolve_manifest.py): `src/workbuddy_bench/runner/resolve_manifest.py`
- [Task preparation](https://github.com/tencent/workbuddy-bench/blob/main/src/workbuddy_bench/runner/prepare_tasks.py): `src/workbuddy_bench/runner/prepare_tasks.py`
- **Benchmark subsets**

### Trial execution

- [Trial orchestration](https://github.com/tencent/workbuddy-bench/blob/main/src/workbuddy_bench/runner/sharded_eval.py): `src/workbuddy_bench/runner/sharded_eval.py`
- [Docker sandbox](https://github.com/tencent/workbuddy-bench/blob/main/src/workbuddy_bench/runner/docker_environment.py): `src/workbuddy_bench/runner/docker_environment.py`
- [Agent harness adapters](https://github.com/tencent/workbuddy-bench/blob/main/src/workbuddy_bench/runner/harness_adapters.py): `src/workbuddy_bench/runner/harness_adapters.py`
- [Agent usage capture](https://github.com/tencent/workbuddy-bench/blob/main/src/workbuddy_bench/agents/_agent_user.py): `src/workbuddy_bench/agents/_agent_user.py`

### Model traffic proxy

- [Proxy routes](https://github.com/tencent/workbuddy-bench/blob/main/src/workbuddy_bench/proxy/config.py): `src/workbuddy_bench/proxy/config.py`
- [Proxy request entry](https://github.com/tencent/workbuddy-bench/blob/main/src/workbuddy_bench/proxy/main.py): `src/workbuddy_bench/proxy/main.py`
- [Protocol and interceptors](https://github.com/tencent/workbuddy-bench/blob/main/src/workbuddy_bench/proxy/pipeline.py): `src/workbuddy_bench/proxy/pipeline.py`
- [Anthropic/OpenAI mapping](https://github.com/tencent/workbuddy-bench/blob/main/src/workbuddy_bench/proxy/protocols/a2o.py): `src/workbuddy_bench/proxy/protocols/a2o.py`
- [Request logging](https://github.com/tencent/workbuddy-bench/blob/main/src/workbuddy_bench/proxy/interceptors/logger.py): `src/workbuddy_bench/proxy/interceptors/logger.py`

### Verification and scoring

- [Judge profile dispatch](https://github.com/tencent/workbuddy-bench/blob/main/src/workbuddy_bench/judge/profiles/dispatcher.py): `src/workbuddy_bench/judge/profiles/dispatcher.py`
- [Composite verifier engine](https://github.com/tencent/workbuddy-bench/blob/main/src/workbuddy_bench/judge/core/engine.py): `src/workbuddy_bench/judge/core/engine.py`
- [Pytest and rule judges](https://github.com/tencent/workbuddy-bench/blob/main/src/workbuddy_bench/judge/runners/harbor_pytest.py): `src/workbuddy_bench/judge/runners/harbor_pytest.py`
- [Artifact inspection](https://github.com/tencent/workbuddy-bench/blob/main/src/workbuddy_bench/judge/evidence/artifact_inspector.py): `src/workbuddy_bench/judge/evidence/artifact_inspector.py`
- [Command execution](https://github.com/tencent/workbuddy-bench/blob/main/src/workbuddy_bench/judge/runtime/command.py): `src/workbuddy_bench/judge/runtime/command.py`
- [Correctness scoring](https://github.com/tencent/workbuddy-bench/blob/main/src/workbuddy_bench/scorer/scorer.py): `src/workbuddy_bench/scorer/scorer.py`
- [LLM post-judge](https://github.com/tencent/workbuddy-bench/blob/main/src/workbuddy_bench/scorer/llm_judge.py): `src/workbuddy_bench/scorer/llm_judge.py`
- **Evaluation results**

### Other components

- **Benchmark operator**
- **Agent CLI**
- **Model backend**

## Connections

- Benchmark operator → Job and model config: configures (evidence: `README.md`)
- Job and model config → Run manifest: resolves (evidence: `src/workbuddy_bench/runner/resolve_manifest.py`)
- Run manifest → Task preparation: prepares tasks
- Benchmark subsets → Task preparation: supplies tasks (evidence: `README.md`)
- Run manifest → Trial orchestration: launches trials
- Trial orchestration → Docker sandbox: runs in (evidence: `README.md`)
- Trial orchestration → Agent harness adapters: dispatches (evidence: `README.md`)
- Agent harness adapters → Agent CLI: runs (evidence: `README.md`)
- Agent CLI → Proxy request entry: sends model requests
- Agent usage capture → Trial orchestration: captures usage
- Run manifest → Proxy routes: configures routes
- Proxy request entry → Protocol and interceptors: dispatches requests (evidence: `src/workbuddy_bench/proxy/main.py`)
- Protocol and interceptors → Proxy routes: uses routes (evidence: `src/workbuddy_bench/proxy/pipeline.py`)
- Protocol and interceptors → Anthropic/OpenAI mapping: converts protocol (evidence: `src/workbuddy_bench/proxy/pipeline.py`)
- Protocol and interceptors → Request logging: logs exchanges (evidence: `src/workbuddy_bench/proxy/pipeline.py`)
- Protocol and interceptors → Model backend: forwards requests (evidence: `src/workbuddy_bench/proxy/pipeline.py`)
- Trial orchestration → Judge profile dispatch: routes judging
- Judge profile dispatch → Composite verifier engine: dispatches profile
- Composite verifier engine → Pytest and rule judges: runs judges
- Pytest and rule judges → Command execution: executes commands (evidence: `src/workbuddy_bench/judge/runners/harbor_pytest.py`)
- Composite verifier engine → Artifact inspection: collects evidence
- Trial orchestration → Correctness scoring: scores trial (evidence: `README.md`)
- Correctness scoring → Evaluation results: writes scores (evidence: `README.md`)
- Evaluation results → LLM post-judge: post-processes (evidence: `src/workbuddy_bench/scorer/llm_judge.py`)
- LLM post-judge → Evaluation results: updates scores (evidence: `src/workbuddy_bench/scorer/llm_judge.py`)
