How imlrz/deepresearch-bench-ii works
DeepResearch Bench II is a rubric-based benchmark and lightweight evaluator for deep-research reports. The main runtime workflow loads task rubrics and model reports, extracts text from DOCX or Markdown, evaluates rubric batches with GPT-5.5, and aggregates scores into dimension-specific CSVs. Its scoring covers information recall, analysis, and presentation, including evidence and blocked-reference outcomes. The README documents broader benchmark construction from expert reports, but that construction is not implemented in the sampled runtime files; links and capabilities beyond those excerpts are represented only where documented.
Main entry points: run_evaluation.py Evaluation Runner · aggregate_scores.py Score Aggregation · tasks_and_rubrics.jsonl Task Rubrics · gpt_client.py GPT Client
Components 11
Evaluation
- Task Rubrics
tasks_and_rubrics.jsonl - Research Reports
- Evaluation Runner
run_evaluation.py - Text Extraction
run_evaluation.py
LLM Judging
- Batch Scoring
run_evaluation.py - GPT Client
gpt_client.py
Score Reporting
- Scored Results
- Score Aggregation
aggregate_scores.py - Dimension CSVs
Other components
- Evaluator
- OpenAI API
Connections 11
- Evaluator → Evaluation Runner: runs evaluation
README.md - Task Rubrics → Evaluation Runner: supplies rubrics
README.md - Research Reports → Evaluation Runner: provides reports
README.md - Evaluation Runner → Text Extraction: extracts text
run_evaluation.py - Evaluation Runner → Batch Scoring: dispatches batches
run_evaluation.py - Batch Scoring → GPT Client: queries
run_evaluation.py - GPT Client → OpenAI API: sends requests
gpt_client.py - Evaluation Runner → Scored Results: writes scores
README.md - Scored Results → Score Aggregation: reads results
aggregate_scores.py - Task Rubrics → Score Aggregation: provides metadata
aggregate_scores.py - Score Aggregation → Dimension CSVs: writes CSVs
aggregate_scores.py