How researchbites/reviewbench works
ReviewBench fetches venue material, canonicalizes it into papers and human comments, generates AI reviews, extracts paper claims, assesses comments against claims, and scores comparisons across human and AI sources. PostgreSQL is the shared state layer for these stages. The README documents the end-to-end pipeline and scoring capabilities; source excerpts and the index confirm selected calls, while some venue parsers, generator variants, and assessment details are represented at their documented subsystem paths rather than inferred internals.
Main entry points: cli.py CLI workflow · queries.py Database queries · generators AI review generators · assess_comments.py Comment assessment
Components 24
Venue ingestion
- Venue fetchers
src/fetchers - Venue parsers
src/parsers - Review comment splitting
src/lib/comment_parser.py
Review generation
- AI review generators
src/generators - Gemini client
src/lib/gemini.py - OpenAI client
src/lib/openai.py - LLM JSON repair
src/lib/json_repair.py
Claims and assessment
- Claim extraction
src/processor/extract_claims.py - Comment assessment
src/processor/assess_comments.py - Assessment types
src/processor/types.py
Scoring and outputs
- Score orchestration
src/processor/score.py - Benchmark metrics
src/processor/metrics.py - Statistical tests
src/processor/statistics.py - Figures and CSVs
src/processor/figures.py
Shared data
- Raw venue data
src/db/schema.py - Papers and comments
src/db/schema.py - Extracted claims
src/db/schema.py - Assessments
src/db/schema.py - Database queries
src/db/queries.py - Postgres connection pool
src/db/client.py - Cloud SQL PostgreSQL
Other components
- Benchmark operator
- CLI workflow
cli.py - Venue APIs and files
Connections 30
- Benchmark operator → CLI workflow: runs workflow
cli.py - CLI workflow → Venue fetchers: selects fetcher
cli.py - Venue fetchers → Venue APIs and files: fetches records
src/fetchers/openreview.py - Venue fetchers → Database queries: stores raw data
src/fetchers/openreview.py - CLI workflow → Venue parsers: selects parser
cli.py - Venue parsers → Database queries: reads raw data
src/parsers/openreview.py - Venue parsers → Review comment splitting: splits reviews
src/parsers/openreview.py - Venue parsers → Database queries: writes papers and comments
src/parsers/openreview.py - Database queries → Postgres connection pool: uses connections
src/db/queries.py - Postgres connection pool → Cloud SQL PostgreSQL: queries database
README.md - CLI workflow → AI review generators: runs generation
cli.py - AI review generators → Database queries: reads papers and saves comments
src/generators/control_gpt.py - AI review generators → Gemini client: calls Gemini
src/generators/control_gemini.py - AI review generators → OpenAI client: calls OpenAI
src/generators/control_gpt.py - AI review generators → LLM JSON repair: repairs output
src/generators/control_gpt.py - CLI workflow → Claim extraction: runs extraction
cli.py - Claim extraction → Database queries: reads papers and saves claims
src/processor/extract_claims.py - Claim extraction → Gemini client: extracts with Gemini
src/processor/extract_claims.py - Claim extraction → LLM JSON repair: parses model output
src/processor/extract_claims.py - CLI workflow → Comment assessment: runs assessment
cli.py - Comment assessment → Database queries: loads inputs and saves assessments
src/processor/assess_comments.py - Comment assessment → Gemini client: assesses with Gemini
src/processor/assess_comments.py - Comment assessment → LLM JSON repair: parses model output
src/processor/assess_comments.py - Comment assessment → Assessment types: uses constants
src/processor/assess_comments.py - CLI workflow → Score orchestration: runs scoring
cli.py - Score orchestration → Benchmark metrics: computes metrics
src/processor/score.py - Score orchestration → Statistical tests: runs statistics
src/processor/score.py - Score orchestration → Figures and CSVs: generates figures
src/processor/score.py - Benchmark metrics → Database queries: loads benchmark data
README.md - Statistical tests → Benchmark metrics: analyzes metric results
src/processor/statistics.py