# imlrz/deepresearch-bench-ii architecture

> Architecture diagram of the GitHub repository imlrz/deepresearch-bench-ii, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-10-02). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/imlrz/deepresearch-bench-ii
- Repository: https://github.com/imlrz/deepresearch-bench-ii

## Overview

DeepResearch Bench II is a rubric-based benchmark and lightweight evaluator for deep-research reports. The main runtime workflow loads task rubrics and model reports, extracts text from DOCX or Markdown, evaluates rubric batches with GPT-5.5, and aggregates scores into dimension-specific CSVs. Its scoring covers information recall, analysis, and presentation, including evidence and blocked-reference outcomes. The README documents broader benchmark construction from expert reports, but that construction is not implemented in the sampled runtime files; links and capabilities beyond those excerpts are represented only where documented.

Main entry points: [`run_evaluation.py`](https://github.com/imlrz/deepresearch-bench-ii/blob/main/run_evaluation.py) (Evaluation Runner), [`aggregate_scores.py`](https://github.com/imlrz/deepresearch-bench-ii/blob/main/aggregate_scores.py) (Score Aggregation), [`tasks_and_rubrics.jsonl`](https://github.com/imlrz/deepresearch-bench-ii/blob/main/tasks_and_rubrics.jsonl) (Task Rubrics), [`gpt_client.py`](https://github.com/imlrz/deepresearch-bench-ii/blob/main/gpt_client.py) (GPT Client)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_evaluation["Evaluation"]
  node_task_data["Task Rubrics"]
  node_reports["Research Reports"]
  node_runner["Evaluation Runner<br/>[run_evaluation.py]"]
  node_text_extract["Text Extraction<br/>[run_evaluation.py]"]
end

subgraph group_judging["LLM Judging"]
  node_batch_judge["Batch Scoring<br/>[run_evaluation.py]"]
  node_gpt_client["GPT Client<br/>[gpt_client.py]"]
end

subgraph group_reporting["Score Reporting"]
  node_score_results["Scored Results"]
  node_aggregator["Score Aggregation"]
  node_csv_reports["Dimension CSVs"]
end

node_researcher(("Evaluator"))
node_openai{{"OpenAI API"}}

node_researcher -->|"runs evaluation"| node_runner
node_task_data -->|"supplies rubrics"| node_runner
node_reports -->|"provides reports"| node_runner
node_runner -->|"extracts text"| node_text_extract
node_runner -->|"dispatches batches"| node_batch_judge
node_batch_judge -->|"queries"| node_gpt_client
node_gpt_client -->|"sends requests"| node_openai
node_runner -->|"writes scores"| node_score_results
node_score_results -->|"reads results"| node_aggregator
node_task_data -->|"provides metadata"| node_aggregator
node_aggregator -->|"writes CSVs"| node_csv_reports

click node_task_data "https://github.com/imlrz/deepresearch-bench-ii/blob/main/tasks_and_rubrics.jsonl"
click node_runner "https://github.com/imlrz/deepresearch-bench-ii/blob/main/run_evaluation.py"
click node_text_extract "https://github.com/imlrz/deepresearch-bench-ii/blob/main/run_evaluation.py"
click node_batch_judge "https://github.com/imlrz/deepresearch-bench-ii/blob/main/run_evaluation.py"
click node_gpt_client "https://github.com/imlrz/deepresearch-bench-ii/blob/main/gpt_client.py"
click node_aggregator "https://github.com/imlrz/deepresearch-bench-ii/blob/main/aggregate_scores.py"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_task_data,node_reports,node_runner,node_text_extract toneBlue
class node_batch_judge,node_gpt_client toneAmber
class node_score_results,node_aggregator,node_csv_reports,node_openai toneMint
class node_researcher toneIndigo
```

## Components

### Evaluation

- [Task Rubrics](https://github.com/imlrz/deepresearch-bench-ii/blob/main/tasks_and_rubrics.jsonl): `tasks_and_rubrics.jsonl`
- **Research Reports**
- [Evaluation Runner](https://github.com/imlrz/deepresearch-bench-ii/blob/main/run_evaluation.py): `run_evaluation.py`
- [Text Extraction](https://github.com/imlrz/deepresearch-bench-ii/blob/main/run_evaluation.py): `run_evaluation.py`

### LLM Judging

- [Batch Scoring](https://github.com/imlrz/deepresearch-bench-ii/blob/main/run_evaluation.py): `run_evaluation.py`
- [GPT Client](https://github.com/imlrz/deepresearch-bench-ii/blob/main/gpt_client.py): `gpt_client.py`

### Score Reporting

- **Scored Results**
- [Score Aggregation](https://github.com/imlrz/deepresearch-bench-ii/blob/main/aggregate_scores.py): `aggregate_scores.py`
- **Dimension CSVs**

### Other components

- **Evaluator**
- **OpenAI API**

## Connections

- Evaluator → Evaluation Runner: runs evaluation (evidence: `README.md`)
- Task Rubrics → Evaluation Runner: supplies rubrics (evidence: `README.md`)
- Research Reports → Evaluation Runner: provides reports (evidence: `README.md`)
- Evaluation Runner → Text Extraction: extracts text (evidence: `run_evaluation.py`)
- Evaluation Runner → Batch Scoring: dispatches batches (evidence: `run_evaluation.py`)
- Batch Scoring → GPT Client: queries (evidence: `run_evaluation.py`)
- GPT Client → OpenAI API: sends requests (evidence: `gpt_client.py`)
- Evaluation Runner → Scored Results: writes scores (evidence: `README.md`)
- Scored Results → Score Aggregation: reads results (evidence: `aggregate_scores.py`)
- Task Rubrics → Score Aggregation: provides metadata (evidence: `aggregate_scores.py`)
- Score Aggregation → Dimension CSVs: writes CSVs (evidence: `aggregate_scores.py`)
