# researchbites/reviewbench architecture

> Architecture diagram of the GitHub repository researchbites/reviewbench, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-10-07). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/researchbites/reviewbench
- Repository: https://github.com/researchbites/reviewbench

## Overview

ReviewBench fetches venue material, canonicalizes it into papers and human comments, generates AI reviews, extracts paper claims, assesses comments against claims, and scores comparisons across human and AI sources. PostgreSQL is the shared state layer for these stages. The README documents the end-to-end pipeline and scoring capabilities; source excerpts and the index confirm selected calls, while some venue parsers, generator variants, and assessment details are represented at their documented subsystem paths rather than inferred internals.

Main entry points: [`cli.py`](https://github.com/researchbites/reviewbench/blob/main/cli.py) (CLI workflow), [`src/db/queries.py`](https://github.com/researchbites/reviewbench/blob/main/src/db/queries.py) (Database queries), [`src/generators`](https://github.com/researchbites/reviewbench/tree/main/src/generators) (AI review generators), [`src/processor/assess_comments.py`](https://github.com/researchbites/reviewbench/blob/main/src/processor/assess_comments.py) (Comment assessment)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_ingest["Venue ingestion"]
  node_fetchers["Venue fetchers"]
  node_parsers["Venue parsers"]
  node_comment_splitter["Review comment splitting<br/>[comment_parser.py]"]
end

subgraph group_generation["Review generation"]
  node_generators["AI review generators"]
  node_gemini_client["Gemini client<br/>[gemini.py]"]
  node_openai_client["OpenAI client<br/>[openai.py]"]
  node_json_repair["LLM JSON repair<br/>[json_repair.py]"]
end

subgraph group_assessment["Claims and assessment"]
  node_claims["Claim extraction<br/>[extract_claims.py]"]
  node_assessments["Comment assessment<br/>[assess_comments.py]"]
  node_processor_types["Assessment types<br/>[types.py]"]
end

subgraph group_scoring["Scoring and outputs"]
  node_scoring["Score orchestration<br/>[score.py]"]
  node_metrics["Benchmark metrics<br/>[metrics.py]"]
  node_statistics["Statistical tests<br/>[statistics.py]"]
  node_figures["Figures and CSVs<br/>[figures.py]"]
end

subgraph group_storage["Shared data"]
  node_raw_data[("Raw venue data<br/>[schema.py]")]
  node_canonical_data[("Papers and comments<br/>[schema.py]")]
  node_claim_data[("Extracted claims<br/>[schema.py]")]
  node_assessment_data[("Assessments<br/>[schema.py]")]
  node_queries["Database queries<br/>[queries.py]"]
  node_db_client["Postgres connection pool<br/>[client.py]"]
  node_postgres[("Cloud SQL PostgreSQL")]
end

node_operator(("Benchmark operator"))
node_cli["CLI workflow<br/>[cli.py]"]
node_venue_services["Venue APIs and files"]

node_operator -->|"runs workflow"| node_cli
node_cli -->|"selects fetcher"| node_fetchers
node_fetchers -->|"fetches records"| node_venue_services
node_fetchers -->|"stores raw data"| node_queries
node_cli -->|"selects parser"| node_parsers
node_parsers -->|"reads raw data"| node_queries
node_parsers -->|"splits reviews"| node_comment_splitter
node_parsers -->|"writes papers and comments"| node_queries
node_queries -->|"uses connections"| node_db_client
node_db_client -.->|"queries database"| node_postgres
node_cli -->|"runs generation"| node_generators
node_generators -->|"reads papers and saves comments"| node_queries
node_generators -->|"calls Gemini"| node_gemini_client
node_generators -->|"calls OpenAI"| node_openai_client
node_generators -->|"repairs output"| node_json_repair
node_cli -->|"runs extraction"| node_claims
node_claims -->|"reads papers and saves claims"| node_queries
node_claims -->|"extracts with Gemini"| node_gemini_client
node_claims -->|"parses model output"| node_json_repair
node_cli -->|"runs assessment"| node_assessments
node_assessments -->|"loads inputs and saves assessments"| node_queries
node_assessments -->|"assesses with Gemini"| node_gemini_client
node_assessments -->|"parses model output"| node_json_repair
node_assessments -->|"uses constants"| node_processor_types
node_cli -->|"runs scoring"| node_scoring
node_scoring -->|"computes metrics"| node_metrics
node_scoring -->|"runs statistics"| node_statistics
node_scoring -->|"generates figures"| node_figures
node_metrics -->|"loads benchmark data"| node_queries
node_statistics -->|"analyzes metric results"| node_metrics

click node_cli "https://github.com/researchbites/reviewbench/blob/main/cli.py"
click node_fetchers "https://github.com/researchbites/reviewbench/tree/main/src/fetchers"
click node_raw_data "https://github.com/researchbites/reviewbench/blob/main/src/db/schema.py"
click node_parsers "https://github.com/researchbites/reviewbench/tree/main/src/parsers"
click node_comment_splitter "https://github.com/researchbites/reviewbench/blob/main/src/lib/comment_parser.py"
click node_canonical_data "https://github.com/researchbites/reviewbench/blob/main/src/db/schema.py"
click node_generators "https://github.com/researchbites/reviewbench/tree/main/src/generators"
click node_gemini_client "https://github.com/researchbites/reviewbench/blob/main/src/lib/gemini.py"
click node_openai_client "https://github.com/researchbites/reviewbench/blob/main/src/lib/openai.py"
click node_json_repair "https://github.com/researchbites/reviewbench/blob/main/src/lib/json_repair.py"
click node_claims "https://github.com/researchbites/reviewbench/blob/main/src/processor/extract_claims.py"
click node_assessments "https://github.com/researchbites/reviewbench/blob/main/src/processor/assess_comments.py"
click node_processor_types "https://github.com/researchbites/reviewbench/blob/main/src/processor/types.py"
click node_claim_data "https://github.com/researchbites/reviewbench/blob/main/src/db/schema.py"
click node_assessment_data "https://github.com/researchbites/reviewbench/blob/main/src/db/schema.py"
click node_queries "https://github.com/researchbites/reviewbench/blob/main/src/db/queries.py"
click node_db_client "https://github.com/researchbites/reviewbench/blob/main/src/db/client.py"
click node_scoring "https://github.com/researchbites/reviewbench/blob/main/src/processor/score.py"
click node_metrics "https://github.com/researchbites/reviewbench/blob/main/src/processor/metrics.py"
click node_statistics "https://github.com/researchbites/reviewbench/blob/main/src/processor/statistics.py"
click node_figures "https://github.com/researchbites/reviewbench/blob/main/src/processor/figures.py"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_fetchers,node_parsers,node_comment_splitter toneBlue
class node_generators,node_gemini_client,node_openai_client,node_json_repair toneAmber
class node_claims,node_assessments,node_processor_types,node_venue_services toneMint
class node_scoring,node_metrics,node_statistics,node_figures toneRose
class node_raw_data,node_canonical_data,node_claim_data,node_assessment_data,node_queries,node_db_client,node_postgres,node_operator toneIndigo
class node_cli toneTeal
```

## Components

### Venue ingestion

- [Venue fetchers](https://github.com/researchbites/reviewbench/tree/main/src/fetchers): `src/fetchers`
- [Venue parsers](https://github.com/researchbites/reviewbench/tree/main/src/parsers): `src/parsers`
- [Review comment splitting](https://github.com/researchbites/reviewbench/blob/main/src/lib/comment_parser.py): `src/lib/comment_parser.py`

### Review generation

- [AI review generators](https://github.com/researchbites/reviewbench/tree/main/src/generators): `src/generators`
- [Gemini client](https://github.com/researchbites/reviewbench/blob/main/src/lib/gemini.py): `src/lib/gemini.py`
- [OpenAI client](https://github.com/researchbites/reviewbench/blob/main/src/lib/openai.py): `src/lib/openai.py`
- [LLM JSON repair](https://github.com/researchbites/reviewbench/blob/main/src/lib/json_repair.py): `src/lib/json_repair.py`

### Claims and assessment

- [Claim extraction](https://github.com/researchbites/reviewbench/blob/main/src/processor/extract_claims.py): `src/processor/extract_claims.py`
- [Comment assessment](https://github.com/researchbites/reviewbench/blob/main/src/processor/assess_comments.py): `src/processor/assess_comments.py`
- [Assessment types](https://github.com/researchbites/reviewbench/blob/main/src/processor/types.py): `src/processor/types.py`

### Scoring and outputs

- [Score orchestration](https://github.com/researchbites/reviewbench/blob/main/src/processor/score.py): `src/processor/score.py`
- [Benchmark metrics](https://github.com/researchbites/reviewbench/blob/main/src/processor/metrics.py): `src/processor/metrics.py`
- [Statistical tests](https://github.com/researchbites/reviewbench/blob/main/src/processor/statistics.py): `src/processor/statistics.py`
- [Figures and CSVs](https://github.com/researchbites/reviewbench/blob/main/src/processor/figures.py): `src/processor/figures.py`

### Shared data

- [Raw venue data](https://github.com/researchbites/reviewbench/blob/main/src/db/schema.py): `src/db/schema.py`
- [Papers and comments](https://github.com/researchbites/reviewbench/blob/main/src/db/schema.py): `src/db/schema.py`
- [Extracted claims](https://github.com/researchbites/reviewbench/blob/main/src/db/schema.py): `src/db/schema.py`
- [Assessments](https://github.com/researchbites/reviewbench/blob/main/src/db/schema.py): `src/db/schema.py`
- [Database queries](https://github.com/researchbites/reviewbench/blob/main/src/db/queries.py): `src/db/queries.py`
- [Postgres connection pool](https://github.com/researchbites/reviewbench/blob/main/src/db/client.py): `src/db/client.py`
- **Cloud SQL PostgreSQL**

### Other components

- **Benchmark operator**
- [CLI workflow](https://github.com/researchbites/reviewbench/blob/main/cli.py): `cli.py`
- **Venue APIs and files**

## Connections

- Benchmark operator → CLI workflow: runs workflow (evidence: `cli.py`)
- CLI workflow → Venue fetchers: selects fetcher (evidence: `cli.py`)
- Venue fetchers → Venue APIs and files: fetches records (evidence: `src/fetchers/openreview.py`)
- Venue fetchers → Database queries: stores raw data (evidence: `src/fetchers/openreview.py`)
- CLI workflow → Venue parsers: selects parser (evidence: `cli.py`)
- Venue parsers → Database queries: reads raw data (evidence: `src/parsers/openreview.py`)
- Venue parsers → Review comment splitting: splits reviews (evidence: `src/parsers/openreview.py`)
- Venue parsers → Database queries: writes papers and comments (evidence: `src/parsers/openreview.py`)
- Database queries → Postgres connection pool: uses connections (evidence: `src/db/queries.py`)
- Postgres connection pool → Cloud SQL PostgreSQL: queries database (evidence: `README.md`)
- CLI workflow → AI review generators: runs generation (evidence: `cli.py`)
- AI review generators → Database queries: reads papers and saves comments (evidence: `src/generators/control_gpt.py`)
- AI review generators → Gemini client: calls Gemini (evidence: `src/generators/control_gemini.py`)
- AI review generators → OpenAI client: calls OpenAI (evidence: `src/generators/control_gpt.py`)
- AI review generators → LLM JSON repair: repairs output (evidence: `src/generators/control_gpt.py`)
- CLI workflow → Claim extraction: runs extraction (evidence: `cli.py`)
- Claim extraction → Database queries: reads papers and saves claims (evidence: `src/processor/extract_claims.py`)
- Claim extraction → Gemini client: extracts with Gemini (evidence: `src/processor/extract_claims.py`)
- Claim extraction → LLM JSON repair: parses model output (evidence: `src/processor/extract_claims.py`)
- CLI workflow → Comment assessment: runs assessment (evidence: `cli.py`)
- Comment assessment → Database queries: loads inputs and saves assessments (evidence: `src/processor/assess_comments.py`)
- Comment assessment → Gemini client: assesses with Gemini (evidence: `src/processor/assess_comments.py`)
- Comment assessment → LLM JSON repair: parses model output (evidence: `src/processor/assess_comments.py`)
- Comment assessment → Assessment types: uses constants (evidence: `src/processor/assess_comments.py`)
- CLI workflow → Score orchestration: runs scoring (evidence: `cli.py`)
- Score orchestration → Benchmark metrics: computes metrics (evidence: `src/processor/score.py`)
- Score orchestration → Statistical tests: runs statistics (evidence: `src/processor/score.py`)
- Score orchestration → Figures and CSVs: generates figures (evidence: `src/processor/score.py`)
- Benchmark metrics → Database queries: loads benchmark data (evidence: `README.md`)
- Statistical tests → Benchmark metrics: analyzes metric results (evidence: `src/processor/statistics.py`)
