# frontisai/enterpriseclawbench architecture

> Architecture diagram of the GitHub repository frontisai/enterpriseclawbench, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-09-25). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/frontisai/enterpriseclawbench
- Repository: https://github.com/frontisai/enterpriseclawbench

## Overview

EnterpriseClawBench constructs benchmark tasks from enterprise-agent session traces, runs tasks through an optional sandbox, and evaluates the resulting run directories against artifact rules and semantic rubrics. Its principal workflow proceeds from trace inventory and extraction through task cleaning, enrichment, selection, packaging, and export; an agent then produces responses and artifacts that the evaluation pipeline scores and reports. The graph retains the documented construction stages and the separate run/evaluation branches. Stage ordering and capabilities are documented, but their internal calls were not sampled, so those links reflect the documented workflow rather than inspected stage implementations.

Main entry points: [`sandbox/runner.py`](https://github.com/frontisai/enterpriseclawbench/blob/main/sandbox/runner.py) (Sandbox runner), [`evaluation/enterprise_clawbench_evaluation/judge.py`](https://github.com/frontisai/enterpriseclawbench/blob/main/evaluation/enterprise_clawbench_evaluation/judge.py) (Artifact and rubric judge), [`sandbox/open_sandbox_client.py`](https://github.com/frontisai/enterpriseclawbench/blob/main/sandbox/open_sandbox_client.py) (Sandbox client), [`construction/enterprise_clawbench_construction/session_protocol.py`](https://github.com/frontisai/enterpriseclawbench/blob/main/construction/enterprise_clawbench_construction/session_protocol.py) (Exported tasks)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_construction["Task construction"]
  node_builder["Construction CLI<br/>[cli.py]"]
  node_pipeline["Pipeline runner<br/>[pipeline.py]"]
  node_inventory["Trace inventory"]
  node_extract["Turn extraction"]
  node_segment["Instance segmentation"]
  node_checks["Mechanical checks"]
  node_join["Mechanical join"]
  node_selfcontained["Prompt rewriting"]
  node_rewrite["Prompt rewrite"]
  node_taxonomy["Task taxonomy"]
  node_skills["Skill subclasses"]
  node_deliverables["Deliverable analysis"]
  node_rules["Rule generation<br/>[stage_10_rules.py]"]
  node_rubric["Semantic rubric"]
  node_selection["Candidate selection"]
  node_pack["Task pack writer"]
  node_validate["Pack validation"]
  node_export["Task export<br/>[stage_15_export.py]"]
end

subgraph group_sandbox["Agent execution"]
  node_runner["Sandbox runner<br/>[runner.py]"]
  node_taskio["Task and fixture IO<br/>[task_io.py]"]
  node_sandboxclient["Sandbox client"]
end

subgraph group_evaluation["Run evaluation"]
  node_judgecli["Evaluation CLI<br/>[cli.py]"]
  node_judge["Artifact and rubric judge<br/>[judge.py]"]
  node_evidence["Evidence preparation<br/>[evidence.py]"]
  node_judgeclient["Judge model client<br/>[judge_client.py]"]
  node_report["Evaluation report<br/>[report.py]"]
end

subgraph group_protocol["Run protocol"]
  node_taskpack["Exported tasks"]
  node_runs["Evaluation run files<br/>[run_protocol.py]"]
end

node_rawtraces["Raw sessions"]
node_agent(("Agent system"))
node_judgeapi{{"OpenAI-compatible API"}}

node_builder -->|"starts"| node_pipeline
node_rawtraces -->|"supplies traces"| node_inventory
node_pipeline -->|"runs stages"| node_inventory
node_inventory -->|"passes inventory"| node_extract
node_extract -->|"passes turns"| node_segment
node_segment -->|"passes instances"| node_checks
node_checks -->|"passes checks"| node_join
node_join -->|"passes joined tasks"| node_selfcontained
node_selfcontained -->|"passes tasks"| node_rewrite
node_rewrite -->|"passes prompts"| node_taxonomy
node_taxonomy -->|"passes taxonomy"| node_skills
node_skills -->|"passes skills"| node_deliverables
node_deliverables -->|"passes deliverables"| node_rules
node_rules -->|"passes rules"| node_rubric
node_rubric -->|"passes rubrics"| node_selection
node_selection -->|"passes candidates"| node_pack
node_pack -->|"passes task pack"| node_validate
node_validate -->|"passes validated pack"| node_export
node_export -->|"writes tasks"| node_taskpack
node_taskpack -->|"provides tasks"| node_runner
node_runner -->|"prepares cases"| node_taskio
node_runner -->|"dispatches work"| node_sandboxclient
node_taskio -->|"supplies fixture uploads"| node_sandboxclient
node_sandboxclient -->|"runs agent"| node_agent
node_agent -->|"returns outputs"| node_sandboxclient
node_runner -->|"writes run files"| node_runs
node_runs -->|"provides run data"| node_judgecli
node_taskpack -->|"provides task definitions"| node_judgecli
node_judgecli -->|"invokes judging"| node_judge
node_judge -->|"prepares evidence"| node_evidence
node_judge -->|"requests model judgment"| node_judgeclient
node_judgeclient -->|"sends prompts"| node_judgeapi
node_judgeapi -->|"returns judgments"| node_judgeclient
node_judge -->|"records scores"| node_report

click node_builder "https://github.com/frontisai/enterpriseclawbench/blob/main/construction/enterprise_clawbench_construction/cli.py"
click node_pipeline "https://github.com/frontisai/enterpriseclawbench/blob/main/construction/enterprise_clawbench_construction/construction_core/pipeline.py"
click node_inventory "https://github.com/frontisai/enterpriseclawbench/blob/main/construction/enterprise_clawbench_construction/stages/stage_00_inventory.py"
click node_extract "https://github.com/frontisai/enterpriseclawbench/blob/main/construction/enterprise_clawbench_construction/stages/stage_01_extract_turns.py"
click node_segment "https://github.com/frontisai/enterpriseclawbench/blob/main/construction/enterprise_clawbench_construction/stages/stage_02_segment_instances.py"
click node_checks "https://github.com/frontisai/enterpriseclawbench/blob/main/construction/enterprise_clawbench_construction/stages/stage_03_mechanical_checks.py"
click node_join "https://github.com/frontisai/enterpriseclawbench/blob/main/construction/enterprise_clawbench_construction/stages/stage_04_mechanical_join.py"
click node_selfcontained "https://github.com/frontisai/enterpriseclawbench/blob/main/construction/enterprise_clawbench_construction/stages/stage_05_self_contained.py"
click node_rewrite "https://github.com/frontisai/enterpriseclawbench/blob/main/construction/enterprise_clawbench_construction/stages/stage_06_prompt_rewrite.py"
click node_taxonomy "https://github.com/frontisai/enterpriseclawbench/blob/main/construction/enterprise_clawbench_construction/stages/stage_07_taxonomy.py"
click node_skills "https://github.com/frontisai/enterpriseclawbench/blob/main/construction/enterprise_clawbench_construction/stages/stage_08_skill_subclass.py"
click node_deliverables "https://github.com/frontisai/enterpriseclawbench/blob/main/construction/enterprise_clawbench_construction/stages/stage_09_deliverables.py"
click node_rules "https://github.com/frontisai/enterpriseclawbench/blob/main/construction/enterprise_clawbench_construction/stages/stage_10_rules.py"
click node_rubric "https://github.com/frontisai/enterpriseclawbench/blob/main/construction/enterprise_clawbench_construction/stages/stage_11_semantic_rubric.py"
click node_selection "https://github.com/frontisai/enterpriseclawbench/blob/main/construction/enterprise_clawbench_construction/stages/stage_12_candidate_selection.py"
click node_pack "https://github.com/frontisai/enterpriseclawbench/blob/main/construction/enterprise_clawbench_construction/stages/stage_13_pack_writer.py"
click node_validate "https://github.com/frontisai/enterpriseclawbench/blob/main/construction/enterprise_clawbench_construction/stages/stage_14_pack_validate.py"
click node_export "https://github.com/frontisai/enterpriseclawbench/blob/main/construction/enterprise_clawbench_construction/stages/stage_15_export.py"
click node_taskpack "https://github.com/frontisai/enterpriseclawbench/blob/main/construction/enterprise_clawbench_construction/session_protocol.py"
click node_runner "https://github.com/frontisai/enterpriseclawbench/blob/main/sandbox/runner.py"
click node_taskio "https://github.com/frontisai/enterpriseclawbench/blob/main/sandbox/task_io.py"
click node_sandboxclient "https://github.com/frontisai/enterpriseclawbench/blob/main/sandbox/open_sandbox_client.py"
click node_runs "https://github.com/frontisai/enterpriseclawbench/blob/main/evaluation/enterprise_clawbench_evaluation/run_protocol.py"
click node_judgecli "https://github.com/frontisai/enterpriseclawbench/blob/main/evaluation/enterprise_clawbench_evaluation/cli.py"
click node_judge "https://github.com/frontisai/enterpriseclawbench/blob/main/evaluation/enterprise_clawbench_evaluation/judge.py"
click node_evidence "https://github.com/frontisai/enterpriseclawbench/blob/main/evaluation/enterprise_clawbench_evaluation/evidence.py"
click node_judgeclient "https://github.com/frontisai/enterpriseclawbench/blob/main/evaluation/enterprise_clawbench_evaluation/judge_client.py"
click node_report "https://github.com/frontisai/enterpriseclawbench/blob/main/evaluation/enterprise_clawbench_evaluation/report.py"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_builder,node_pipeline,node_inventory,node_extract,node_segment,node_checks,node_join,node_selfcontained,node_rewrite,node_taxonomy,node_skills,node_deliverables,node_rules,node_rubric,node_selection,node_pack,node_validate,node_export toneBlue
class node_runner,node_taskio,node_sandboxclient toneAmber
class node_judgecli,node_judge,node_evidence,node_judgeclient,node_report,node_judgeapi toneMint
class node_taskpack,node_runs toneRose
class node_rawtraces,node_agent toneIndigo
```

## Components

### Task construction

- [Construction CLI](https://github.com/frontisai/enterpriseclawbench/blob/main/construction/enterprise_clawbench_construction/cli.py): `construction/enterprise_clawbench_construction/cli.py`
- [Pipeline runner](https://github.com/frontisai/enterpriseclawbench/blob/main/construction/enterprise_clawbench_construction/construction_core/pipeline.py): `construction/enterprise_clawbench_construction/construction_core/pipeline.py`
- [Trace inventory](https://github.com/frontisai/enterpriseclawbench/blob/main/construction/enterprise_clawbench_construction/stages/stage_00_inventory.py): `construction/enterprise_clawbench_construction/stages/stage_00_inventory.py`
- [Turn extraction](https://github.com/frontisai/enterpriseclawbench/blob/main/construction/enterprise_clawbench_construction/stages/stage_01_extract_turns.py): `construction/enterprise_clawbench_construction/stages/stage_01_extract_turns.py`
- [Instance segmentation](https://github.com/frontisai/enterpriseclawbench/blob/main/construction/enterprise_clawbench_construction/stages/stage_02_segment_instances.py): `construction/enterprise_clawbench_construction/stages/stage_02_segment_instances.py`
- [Mechanical checks](https://github.com/frontisai/enterpriseclawbench/blob/main/construction/enterprise_clawbench_construction/stages/stage_03_mechanical_checks.py): `construction/enterprise_clawbench_construction/stages/stage_03_mechanical_checks.py`
- [Mechanical join](https://github.com/frontisai/enterpriseclawbench/blob/main/construction/enterprise_clawbench_construction/stages/stage_04_mechanical_join.py): `construction/enterprise_clawbench_construction/stages/stage_04_mechanical_join.py`
- [Prompt rewriting](https://github.com/frontisai/enterpriseclawbench/blob/main/construction/enterprise_clawbench_construction/stages/stage_05_self_contained.py): `construction/enterprise_clawbench_construction/stages/stage_05_self_contained.py`
- [Prompt rewrite](https://github.com/frontisai/enterpriseclawbench/blob/main/construction/enterprise_clawbench_construction/stages/stage_06_prompt_rewrite.py): `construction/enterprise_clawbench_construction/stages/stage_06_prompt_rewrite.py`
- [Task taxonomy](https://github.com/frontisai/enterpriseclawbench/blob/main/construction/enterprise_clawbench_construction/stages/stage_07_taxonomy.py): `construction/enterprise_clawbench_construction/stages/stage_07_taxonomy.py`
- [Skill subclasses](https://github.com/frontisai/enterpriseclawbench/blob/main/construction/enterprise_clawbench_construction/stages/stage_08_skill_subclass.py): `construction/enterprise_clawbench_construction/stages/stage_08_skill_subclass.py`
- [Deliverable analysis](https://github.com/frontisai/enterpriseclawbench/blob/main/construction/enterprise_clawbench_construction/stages/stage_09_deliverables.py): `construction/enterprise_clawbench_construction/stages/stage_09_deliverables.py`
- [Rule generation](https://github.com/frontisai/enterpriseclawbench/blob/main/construction/enterprise_clawbench_construction/stages/stage_10_rules.py): `construction/enterprise_clawbench_construction/stages/stage_10_rules.py`
- [Semantic rubric](https://github.com/frontisai/enterpriseclawbench/blob/main/construction/enterprise_clawbench_construction/stages/stage_11_semantic_rubric.py): `construction/enterprise_clawbench_construction/stages/stage_11_semantic_rubric.py`
- [Candidate selection](https://github.com/frontisai/enterpriseclawbench/blob/main/construction/enterprise_clawbench_construction/stages/stage_12_candidate_selection.py): `construction/enterprise_clawbench_construction/stages/stage_12_candidate_selection.py`
- [Task pack writer](https://github.com/frontisai/enterpriseclawbench/blob/main/construction/enterprise_clawbench_construction/stages/stage_13_pack_writer.py): `construction/enterprise_clawbench_construction/stages/stage_13_pack_writer.py`
- [Pack validation](https://github.com/frontisai/enterpriseclawbench/blob/main/construction/enterprise_clawbench_construction/stages/stage_14_pack_validate.py): `construction/enterprise_clawbench_construction/stages/stage_14_pack_validate.py`
- [Task export](https://github.com/frontisai/enterpriseclawbench/blob/main/construction/enterprise_clawbench_construction/stages/stage_15_export.py): `construction/enterprise_clawbench_construction/stages/stage_15_export.py`

### Agent execution

- [Sandbox runner](https://github.com/frontisai/enterpriseclawbench/blob/main/sandbox/runner.py): `sandbox/runner.py`
- [Task and fixture IO](https://github.com/frontisai/enterpriseclawbench/blob/main/sandbox/task_io.py): `sandbox/task_io.py`
- [Sandbox client](https://github.com/frontisai/enterpriseclawbench/blob/main/sandbox/open_sandbox_client.py): `sandbox/open_sandbox_client.py`

### Run evaluation

- [Evaluation CLI](https://github.com/frontisai/enterpriseclawbench/blob/main/evaluation/enterprise_clawbench_evaluation/cli.py): `evaluation/enterprise_clawbench_evaluation/cli.py`
- [Artifact and rubric judge](https://github.com/frontisai/enterpriseclawbench/blob/main/evaluation/enterprise_clawbench_evaluation/judge.py): `evaluation/enterprise_clawbench_evaluation/judge.py`
- [Evidence preparation](https://github.com/frontisai/enterpriseclawbench/blob/main/evaluation/enterprise_clawbench_evaluation/evidence.py): `evaluation/enterprise_clawbench_evaluation/evidence.py`
- [Judge model client](https://github.com/frontisai/enterpriseclawbench/blob/main/evaluation/enterprise_clawbench_evaluation/judge_client.py): `evaluation/enterprise_clawbench_evaluation/judge_client.py`
- [Evaluation report](https://github.com/frontisai/enterpriseclawbench/blob/main/evaluation/enterprise_clawbench_evaluation/report.py): `evaluation/enterprise_clawbench_evaluation/report.py`

### Run protocol

- [Exported tasks](https://github.com/frontisai/enterpriseclawbench/blob/main/construction/enterprise_clawbench_construction/session_protocol.py): `construction/enterprise_clawbench_construction/session_protocol.py`
- [Evaluation run files](https://github.com/frontisai/enterpriseclawbench/blob/main/evaluation/enterprise_clawbench_evaluation/run_protocol.py): `evaluation/enterprise_clawbench_evaluation/run_protocol.py`

### Other components

- **Raw sessions**
- **Agent system**
- **OpenAI-compatible API**

## Connections

- Construction CLI → Pipeline runner: starts
- Raw sessions → Trace inventory: supplies traces
- Pipeline runner → Trace inventory: runs stages
- Trace inventory → Turn extraction: passes inventory
- Turn extraction → Instance segmentation: passes turns
- Instance segmentation → Mechanical checks: passes instances
- Mechanical checks → Mechanical join: passes checks
- Mechanical join → Prompt rewriting: passes joined tasks
- Prompt rewriting → Prompt rewrite: passes tasks
- Prompt rewrite → Task taxonomy: passes prompts
- Task taxonomy → Skill subclasses: passes taxonomy
- Skill subclasses → Deliverable analysis: passes skills
- Deliverable analysis → Rule generation: passes deliverables
- Rule generation → Semantic rubric: passes rules
- Semantic rubric → Candidate selection: passes rubrics
- Candidate selection → Task pack writer: passes candidates
- Task pack writer → Pack validation: passes task pack
- Pack validation → Task export: passes validated pack
- Task export → Exported tasks: writes tasks
- Exported tasks → Sandbox runner: provides tasks
- Sandbox runner → Task and fixture IO: prepares cases
- Sandbox runner → Sandbox client: dispatches work
- Task and fixture IO → Sandbox client: supplies fixture uploads
- Sandbox client → Agent system: runs agent
- Agent system → Sandbox client: returns outputs
- Sandbox runner → Evaluation run files: writes run files
- Evaluation run files → Evaluation CLI: provides run data
- Exported tasks → Evaluation CLI: provides task definitions
- Evaluation CLI → Artifact and rubric judge: invokes judging
- Artifact and rubric judge → Evidence preparation: prepares evidence
- Artifact and rubric judge → Judge model client: requests model judgment
- Judge model client → OpenAI-compatible API: sends prompts
- OpenAI-compatible API → Judge model client: returns judgments
- Artifact and rubric judge → Evaluation report: records scores