# harveyai/harvey-labs architecture

> Architecture diagram of the GitHub repository harveyai/harvey-labs, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-09-29). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/harveyai/harvey-labs
- Repository: https://github.com/harveyai/harvey-labs
- Video tour (about a minute): https://gitdiagram.com/harveyai/harvey-labs/video

## Overview

Harvey LAB benchmarks legal-work agents using task instructions, documents, and rubrics. Its main workflow is to load a task, run an agent through a provider adapter and bounded workspace tools, then score the run and present results. The repository also includes task discovery, document-production skills, trajectory playback, and model sweeps. Some orchestration and evaluation source was not sampled, so uncertain internal links are omitted; unsampled skill implementations are represented by their existing directories.

Main entry points: [`lab_core/harness/run.py`](https://github.com/harveyai/harvey-labs/blob/main/lab_core/harness/run.py) (Run entrypoint), [`lab_core/root.py`](https://github.com/harveyai/harvey-labs/blob/main/lab_core/root.py) (Benchmark root), [`lab_core/utils/list_tasks.py`](https://github.com/harveyai/harvey-labs/blob/main/lab_core/utils/list_tasks.py) (Task discovery), [`lab_core/harness/agent_loop.py`](https://github.com/harveyai/harvey-labs/blob/main/lab_core/harness/agent_loop.py) (Agent loop)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_task_access["Task access"]
  node_task_catalog["Task discovery<br/>[list_tasks.py]"]
  node_root["Benchmark root<br/>[root.py]"]
end

subgraph group_agent_execution["Agent execution"]
  node_run["Run entrypoint<br/>[run.py]"]
  node_agent_loop["Agent loop<br/>[agent_loop.py]"]
  node_adapters["Provider adapters"]
  node_tools["Workspace tools<br/>[tools.py]"]
  node_sandbox["Task sandbox<br/>[sandbox.py]"]
  node_doc_parser["Document parser<br/>[parse_doc.py]"]
  node_docx_skills["DOCX skills"]
  node_pptx_skills["PPTX skills"]
  node_xlsx_skills["XLSX skills"]
end

subgraph group_evaluation["Evaluation"]
  node_eval_run["Evaluation runner<br/>[run_eval.py]"]
  node_scoring["Rubric scoring<br/>[scoring.py]"]
  node_judge["LLM judge<br/>[judge.py]"]
  node_report["Run report<br/>[report.py]"]
  node_compare["Run comparison<br/>[compare.py]"]
  node_charts["Charts<br/>[charts.py]"]
end

subgraph group_review_tools["Review tools"]
  node_playback["Trajectory playback<br/>[playback.py]"]
  node_sweep["Model sweeps<br/>[sweep.py]"]
end

node_user(("Benchmark user"))
node_task_data[("Task dataset")]
node_provider_apis["Model providers"]

node_user -->|"starts run"| node_run
node_user -->|"lists tasks"| node_task_catalog
node_run -->|"resolves root"| node_root
node_task_catalog -->|"resolves root"| node_root
node_run -->|"loads task"| node_task_data
node_task_catalog -->|"scans tasks"| node_task_data
node_run -->|"creates adapter"| node_adapters
node_adapters -->|"calls API"| node_provider_apis
node_run -->|"starts loop"| node_agent_loop
node_agent_loop -->|"requests response"| node_adapters
node_agent_loop -->|"dispatches tools"| node_tools
node_tools -->|"executes workspace"| node_sandbox
node_sandbox -->|"runs parser"| node_doc_parser
node_eval_run -->|"scores run"| node_scoring
node_scoring -->|"requests judgments"| node_judge
node_report -->|"resolves results"| node_root
node_compare -->|"reads results"| node_root
node_compare -->|"builds charts"| node_charts
node_playback -->|"loads run data"| node_root
node_sweep -->|"launches runs"| node_run
node_sweep -->|"evaluates runs"| node_eval_run

click node_run "https://github.com/harveyai/harvey-labs/blob/main/lab_core/harness/run.py"
click node_task_catalog "https://github.com/harveyai/harvey-labs/blob/main/lab_core/utils/list_tasks.py"
click node_root "https://github.com/harveyai/harvey-labs/blob/main/lab_core/root.py"
click node_agent_loop "https://github.com/harveyai/harvey-labs/blob/main/lab_core/harness/agent_loop.py"
click node_adapters "https://github.com/harveyai/harvey-labs/tree/main/lab_core/harness/adapters"
click node_tools "https://github.com/harveyai/harvey-labs/blob/main/lab_core/harness/tools.py"
click node_sandbox "https://github.com/harveyai/harvey-labs/blob/main/lab_core/sandbox/sandbox.py"
click node_doc_parser "https://github.com/harveyai/harvey-labs/blob/main/lab_core/sandbox/parsers/parse_doc.py"
click node_docx_skills "https://github.com/harveyai/harvey-labs/tree/main/lab_core/harness/skills/docx/scripts"
click node_pptx_skills "https://github.com/harveyai/harvey-labs/tree/main/lab_core/harness/skills/pptx/scripts"
click node_xlsx_skills "https://github.com/harveyai/harvey-labs/tree/main/lab_core/harness/skills/xlsx/scripts"
click node_eval_run "https://github.com/harveyai/harvey-labs/blob/main/lab_core/evaluation/run_eval.py"
click node_scoring "https://github.com/harveyai/harvey-labs/blob/main/lab_core/evaluation/scoring.py"
click node_judge "https://github.com/harveyai/harvey-labs/blob/main/lab_core/evaluation/judge.py"
click node_report "https://github.com/harveyai/harvey-labs/blob/main/lab_core/evaluation/report.py"
click node_compare "https://github.com/harveyai/harvey-labs/blob/main/lab_core/evaluation/compare.py"
click node_charts "https://github.com/harveyai/harvey-labs/blob/main/lab_core/evaluation/charts.py"
click node_playback "https://github.com/harveyai/harvey-labs/blob/main/lab_core/utils/playback.py"
click node_sweep "https://github.com/harveyai/harvey-labs/blob/main/lab_core/utils/sweep.py"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_task_catalog,node_root,node_user toneBlue
class node_run,node_agent_loop,node_adapters,node_tools,node_sandbox,node_doc_parser,node_docx_skills,node_pptx_skills,node_xlsx_skills,node_task_data toneAmber
class node_eval_run,node_scoring,node_judge,node_report,node_compare,node_charts toneMint
class node_playback,node_sweep toneRose
class node_provider_apis toneIndigo
```

## Components

### Task access

- [Task discovery](https://github.com/harveyai/harvey-labs/blob/main/lab_core/utils/list_tasks.py): `lab_core/utils/list_tasks.py`
- [Benchmark root](https://github.com/harveyai/harvey-labs/blob/main/lab_core/root.py): `lab_core/root.py`

### Agent execution

- [Run entrypoint](https://github.com/harveyai/harvey-labs/blob/main/lab_core/harness/run.py): `lab_core/harness/run.py`
- [Agent loop](https://github.com/harveyai/harvey-labs/blob/main/lab_core/harness/agent_loop.py): `lab_core/harness/agent_loop.py`
- [Provider adapters](https://github.com/harveyai/harvey-labs/tree/main/lab_core/harness/adapters): `lab_core/harness/adapters`
- [Workspace tools](https://github.com/harveyai/harvey-labs/blob/main/lab_core/harness/tools.py): `lab_core/harness/tools.py`
- [Task sandbox](https://github.com/harveyai/harvey-labs/blob/main/lab_core/sandbox/sandbox.py): `lab_core/sandbox/sandbox.py`
- [Document parser](https://github.com/harveyai/harvey-labs/blob/main/lab_core/sandbox/parsers/parse_doc.py): `lab_core/sandbox/parsers/parse_doc.py`
- [DOCX skills](https://github.com/harveyai/harvey-labs/tree/main/lab_core/harness/skills/docx/scripts): `lab_core/harness/skills/docx/scripts`
- [PPTX skills](https://github.com/harveyai/harvey-labs/tree/main/lab_core/harness/skills/pptx/scripts): `lab_core/harness/skills/pptx/scripts`
- [XLSX skills](https://github.com/harveyai/harvey-labs/tree/main/lab_core/harness/skills/xlsx/scripts): `lab_core/harness/skills/xlsx/scripts`

### Evaluation

- [Evaluation runner](https://github.com/harveyai/harvey-labs/blob/main/lab_core/evaluation/run_eval.py): `lab_core/evaluation/run_eval.py`
- [Rubric scoring](https://github.com/harveyai/harvey-labs/blob/main/lab_core/evaluation/scoring.py): `lab_core/evaluation/scoring.py`
- [LLM judge](https://github.com/harveyai/harvey-labs/blob/main/lab_core/evaluation/judge.py): `lab_core/evaluation/judge.py`
- [Run report](https://github.com/harveyai/harvey-labs/blob/main/lab_core/evaluation/report.py): `lab_core/evaluation/report.py`
- [Run comparison](https://github.com/harveyai/harvey-labs/blob/main/lab_core/evaluation/compare.py): `lab_core/evaluation/compare.py`
- [Charts](https://github.com/harveyai/harvey-labs/blob/main/lab_core/evaluation/charts.py): `lab_core/evaluation/charts.py`

### Review tools

- [Trajectory playback](https://github.com/harveyai/harvey-labs/blob/main/lab_core/utils/playback.py): `lab_core/utils/playback.py`
- [Model sweeps](https://github.com/harveyai/harvey-labs/blob/main/lab_core/utils/sweep.py): `lab_core/utils/sweep.py`

### Other components

- **Benchmark user**
- **Task dataset**
- **Model providers**

## Connections

- Benchmark user → Run entrypoint: starts run
- Benchmark user → Task discovery: lists tasks
- Run entrypoint → Benchmark root: resolves root
- Task discovery → Benchmark root: resolves root
- Run entrypoint → Task dataset: loads task
- Task discovery → Task dataset: scans tasks
- Run entrypoint → Provider adapters: creates adapter
- Provider adapters → Model providers: calls API
- Run entrypoint → Agent loop: starts loop
- Agent loop → Provider adapters: requests response
- Agent loop → Workspace tools: dispatches tools
- Workspace tools → Task sandbox: executes workspace
- Task sandbox → Document parser: runs parser
- Evaluation runner → Rubric scoring: scores run
- Rubric scoring → LLM judge: requests judgments
- Run report → Benchmark root: resolves results
- Run comparison → Benchmark root: reads results
- Run comparison → Charts: builds charts
- Trajectory playback → Benchmark root: loads run data
- Model sweeps → Run entrypoint: launches runs
- Model sweeps → Evaluation runner: evaluates runs