# adrian-sit/paperprobe architecture

> Architecture diagram of the GitHub repository adrian-sit/paperprobe, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-11-14). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/adrian-sit/paperprobe
- Repository: https://github.com/adrian-sit/paperprobe
- Video tour (about a minute): https://gitdiagram.com/adrian-sit/paperprobe/video

## Overview

PaperProbe ingests research papers from PDFs or OpenReview, preserves revision and source provenance, and produces structured paper context and discussion questions. The main workflow runs through FastAPI routes into ingestion and parsing, persists versioned records, then generates questions through Gemini or a bounded agent workflow; readers can also manage a separate example library. The graph includes storage, retrieval, and request-scoped progress capabilities documented in the README. Some source excerpts are partial, so unsampled implementation links are shown only when the README documents them.

Main entry points: [`app/services/paper_pipeline.py`](https://github.com/adrian-sit/paperprobe/blob/main/app/services/paper_pipeline.py) (Paper pipeline), [`app/services/agent_graph.py`](https://github.com/adrian-sit/paperprobe/blob/main/app/services/agent_graph.py) (Agent workflow), [`app/api/routes/versioned_papers.py`](https://github.com/adrian-sit/paperprobe/blob/main/app/api/routes/versioned_papers.py) (Paper routes), [`app/services/example_retrieval.py`](https://github.com/adrian-sit/paperprobe/blob/main/app/services/example_retrieval.py) (Example retrieval)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_api["API and UI"]
  node_browser["Browser interface<br/>[app.js]"]
  node_paper_routes["Paper routes"]
  node_example_routes["Example routes<br/>[examples.py]"]
  node_progress["Progress SSE<br/>[progress.py]"]
  node_contracts_papers["Paper contracts<br/>[papers.py]"]
  node_contracts_extraction["Extraction contracts<br/>[extraction.py]"]
  node_contracts_examples["Example contracts<br/>[examples.py]"]
end

subgraph group_ingest["Paper ingestion"]
  node_pipeline["Paper pipeline<br/>[paper_pipeline.py]"]
  node_openreview["OpenReview adapter<br/>[openreview.py]"]
  node_archive["Raw source archive"]
  node_pdf["PDF parsing<br/>[pdf.py]"]
  node_sections["Section normalization<br/>[paper_sections.py]"]
end

subgraph group_intelligence["Question intelligence"]
  node_agent["Agent workflow<br/>[agent_graph.py]"]
  node_agent_tools["Agent tools<br/>[agent_tools.py]"]
  node_agent_policy["Critique policy<br/>[agent_policy.py]"]
  node_gemini["Gemini service<br/>[gemini.py]"]
  node_example_library["Example library<br/>[example_library.py]"]
  node_example_upload["Example upload<br/>[example_upload.py]"]
  node_example_retrieval["Example retrieval"]
end

subgraph group_storage["Persistence and retrieval"]
  node_config["Runtime settings<br/>[config.py]"]
  node_session["Async DB sessions<br/>[session.py]"]
  node_models["Relational models<br/>[models.py]"]
  node_postgres[("PostgreSQL / pgvector")]
  node_mongodb[("MongoDB archive")]
end

node_reader(("Reader / client"))
node_external_openreview{{"OpenReview service"}}
node_external_gemini{{"Gemini API"}}

node_reader -->|"uses"| node_browser
node_browser -->|"calls REST"| node_paper_routes
node_reader -->|"calls REST"| node_paper_routes
node_paper_routes -->|"invokes ingestion"| node_pipeline
node_paper_routes -->|"starts workflow"| node_agent
node_paper_routes -->|"streams activity"| node_progress
node_progress -->|"streams results"| node_reader
node_example_routes -->|"manages examples"| node_example_library
node_example_routes -->|"uploads pairs"| node_example_upload
node_pipeline -->|"fetches source"| node_openreview
node_pipeline -->|"parses PDF"| node_pdf
node_pdf -->|"normalizes text"| node_sections
node_pipeline -->|"extracts fields"| node_gemini
node_openreview -->|"fetches notes"| node_external_openreview
node_openreview -->|"archives responses"| node_archive
node_archive -->|"writes snapshots"| node_mongodb
node_pipeline -->|"persists revisions"| node_postgres
node_pipeline -->|"uses sessions"| node_session
node_agent -->|"dispatches tools"| node_agent_tools
node_agent -->|"applies policy"| node_agent_policy
node_agent -->|"critiques questions"| node_gemini
node_agent -->|"retrieves examples"| node_example_retrieval
node_agent -->|"reads and saves"| node_postgres
node_gemini -->|"generates content"| node_external_gemini
node_example_retrieval -->|"embeds context"| node_external_gemini
node_example_retrieval -->|"ranks examples"| node_postgres
node_example_upload -->|"parses paper"| node_pdf
node_example_upload -->|"creates embedding"| node_example_retrieval
node_example_library -->|"stores pairs"| node_postgres
node_session -->|"loads DSN"| node_config
node_session -->|"opens connection"| node_postgres
node_pipeline -->|"returns paper data"| node_contracts_papers
node_gemini -->|"uses typed output"| node_contracts_extraction
node_example_retrieval -->|"returns matches"| node_contracts_examples
node_models -->|"defines schema"| node_postgres

click node_browser "https://github.com/adrian-sit/paperprobe/blob/main/app/web/app.js"
click node_paper_routes "https://github.com/adrian-sit/paperprobe/blob/main/app/api/routes/versioned_papers.py"
click node_example_routes "https://github.com/adrian-sit/paperprobe/blob/main/app/api/routes/examples.py"
click node_progress "https://github.com/adrian-sit/paperprobe/blob/main/app/api/progress.py"
click node_contracts_papers "https://github.com/adrian-sit/paperprobe/blob/main/app/schemas/papers.py"
click node_contracts_extraction "https://github.com/adrian-sit/paperprobe/blob/main/app/schemas/extraction.py"
click node_contracts_examples "https://github.com/adrian-sit/paperprobe/blob/main/app/schemas/examples.py"
click node_pipeline "https://github.com/adrian-sit/paperprobe/blob/main/app/services/paper_pipeline.py"
click node_openreview "https://github.com/adrian-sit/paperprobe/blob/main/app/services/openreview.py"
click node_archive "https://github.com/adrian-sit/paperprobe/blob/main/app/services/openreview_archive.py"
click node_pdf "https://github.com/adrian-sit/paperprobe/blob/main/app/services/pdf.py"
click node_sections "https://github.com/adrian-sit/paperprobe/blob/main/app/services/paper_sections.py"
click node_agent "https://github.com/adrian-sit/paperprobe/blob/main/app/services/agent_graph.py"
click node_agent_tools "https://github.com/adrian-sit/paperprobe/blob/main/app/services/agent_tools.py"
click node_agent_policy "https://github.com/adrian-sit/paperprobe/blob/main/app/services/agent_policy.py"
click node_gemini "https://github.com/adrian-sit/paperprobe/blob/main/app/services/gemini.py"
click node_example_library "https://github.com/adrian-sit/paperprobe/blob/main/app/services/example_library.py"
click node_example_upload "https://github.com/adrian-sit/paperprobe/blob/main/app/services/example_upload.py"
click node_example_retrieval "https://github.com/adrian-sit/paperprobe/blob/main/app/services/example_retrieval.py"
click node_config "https://github.com/adrian-sit/paperprobe/blob/main/app/core/config.py"
click node_session "https://github.com/adrian-sit/paperprobe/blob/main/app/db/session.py"
click node_models "https://github.com/adrian-sit/paperprobe/blob/main/app/db/models.py"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_browser,node_paper_routes,node_example_routes,node_progress,node_contracts_papers,node_contracts_extraction,node_contracts_examples,node_reader,node_external_openreview toneBlue
class node_pipeline,node_openreview,node_archive,node_pdf,node_sections toneAmber
class node_agent,node_agent_tools,node_agent_policy,node_gemini,node_example_library,node_example_upload,node_example_retrieval,node_external_gemini toneMint
class node_config,node_session,node_models,node_postgres,node_mongodb toneRose
```

## Components

### API and UI

- [Browser interface](https://github.com/adrian-sit/paperprobe/blob/main/app/web/app.js): `app/web/app.js`
- [Paper routes](https://github.com/adrian-sit/paperprobe/blob/main/app/api/routes/versioned_papers.py): `app/api/routes/versioned_papers.py`
- [Example routes](https://github.com/adrian-sit/paperprobe/blob/main/app/api/routes/examples.py): `app/api/routes/examples.py`
- [Progress SSE](https://github.com/adrian-sit/paperprobe/blob/main/app/api/progress.py): `app/api/progress.py`
- [Paper contracts](https://github.com/adrian-sit/paperprobe/blob/main/app/schemas/papers.py): `app/schemas/papers.py`
- [Extraction contracts](https://github.com/adrian-sit/paperprobe/blob/main/app/schemas/extraction.py): `app/schemas/extraction.py`
- [Example contracts](https://github.com/adrian-sit/paperprobe/blob/main/app/schemas/examples.py): `app/schemas/examples.py`

### Paper ingestion

- [Paper pipeline](https://github.com/adrian-sit/paperprobe/blob/main/app/services/paper_pipeline.py): `app/services/paper_pipeline.py`
- [OpenReview adapter](https://github.com/adrian-sit/paperprobe/blob/main/app/services/openreview.py): `app/services/openreview.py`
- [Raw source archive](https://github.com/adrian-sit/paperprobe/blob/main/app/services/openreview_archive.py): `app/services/openreview_archive.py`
- [PDF parsing](https://github.com/adrian-sit/paperprobe/blob/main/app/services/pdf.py): `app/services/pdf.py`
- [Section normalization](https://github.com/adrian-sit/paperprobe/blob/main/app/services/paper_sections.py): `app/services/paper_sections.py`

### Question intelligence

- [Agent workflow](https://github.com/adrian-sit/paperprobe/blob/main/app/services/agent_graph.py): `app/services/agent_graph.py`
- [Agent tools](https://github.com/adrian-sit/paperprobe/blob/main/app/services/agent_tools.py): `app/services/agent_tools.py`
- [Critique policy](https://github.com/adrian-sit/paperprobe/blob/main/app/services/agent_policy.py): `app/services/agent_policy.py`
- [Gemini service](https://github.com/adrian-sit/paperprobe/blob/main/app/services/gemini.py): `app/services/gemini.py`
- [Example library](https://github.com/adrian-sit/paperprobe/blob/main/app/services/example_library.py): `app/services/example_library.py`
- [Example upload](https://github.com/adrian-sit/paperprobe/blob/main/app/services/example_upload.py): `app/services/example_upload.py`
- [Example retrieval](https://github.com/adrian-sit/paperprobe/blob/main/app/services/example_retrieval.py): `app/services/example_retrieval.py`

### Persistence and retrieval

- [Runtime settings](https://github.com/adrian-sit/paperprobe/blob/main/app/core/config.py): `app/core/config.py`
- [Async DB sessions](https://github.com/adrian-sit/paperprobe/blob/main/app/db/session.py): `app/db/session.py`
- [Relational models](https://github.com/adrian-sit/paperprobe/blob/main/app/db/models.py): `app/db/models.py`
- **PostgreSQL / pgvector**
- **MongoDB archive**

### Other components

- **Reader / client**
- **OpenReview service**
- **Gemini API**

## Connections

- Reader / client → Browser interface: uses (evidence: `README.md`)
- Browser interface → Paper routes: calls REST (evidence: `README.md`)
- Reader / client → Paper routes: calls REST (evidence: `README.md`)
- Paper routes → Paper pipeline: invokes ingestion (evidence: `app/api/routes/versioned_papers.py`)
- Paper routes → Agent workflow: starts workflow (evidence: `app/api/routes/versioned_papers.py`)
- Paper routes → Progress SSE: streams activity (evidence: `app/api/routes/versioned_papers.py`)
- Progress SSE → Reader / client: streams results (evidence: `README.md`)
- Example routes → Example library: manages examples (evidence: `app/api/routes/examples.py`)
- Example routes → Example upload: uploads pairs (evidence: `app/api/routes/examples.py`)
- Paper pipeline → OpenReview adapter: fetches source (evidence: `app/services/paper_pipeline.py`)
- Paper pipeline → PDF parsing: parses PDF (evidence: `README.md`)
- PDF parsing → Section normalization: normalizes text (evidence: `app/services/pdf.py`)
- Paper pipeline → Gemini service: extracts fields (evidence: `app/services/paper_pipeline.py`)
- OpenReview adapter → OpenReview service: fetches notes (evidence: `README.md`)
- OpenReview adapter → Raw source archive: archives responses (evidence: `app/services/openreview.py`)
- Raw source archive → MongoDB archive: writes snapshots (evidence: `README.md`)
- Paper pipeline → PostgreSQL / pgvector: persists revisions (evidence: `README.md`)
- Paper pipeline → Async DB sessions: uses sessions (evidence: `app/services/paper_pipeline.py`)
- Agent workflow → Agent tools: dispatches tools (evidence: `app/services/agent_graph.py`)
- Agent workflow → Critique policy: applies policy (evidence: `app/services/agent_graph.py`)
- Agent workflow → Gemini service: critiques questions (evidence: `README.md`)
- Agent workflow → Example retrieval: retrieves examples (evidence: `README.md`)
- Agent workflow → PostgreSQL / pgvector: reads and saves (evidence: `README.md`)
- Gemini service → Gemini API: generates content (evidence: `README.md`)
- Example retrieval → Gemini API: embeds context (evidence: `app/services/example_retrieval.py`)
- Example retrieval → PostgreSQL / pgvector: ranks examples (evidence: `app/services/example_retrieval.py`)
- Example upload → PDF parsing: parses paper (evidence: `app/services/example_upload.py`)
- Example upload → Example retrieval: creates embedding (evidence: `app/services/example_upload.py`)
- Example library → PostgreSQL / pgvector: stores pairs (evidence: `app/services/example_library.py`)
- Async DB sessions → Runtime settings: loads DSN (evidence: `app/db/session.py`)
- Async DB sessions → PostgreSQL / pgvector: opens connection (evidence: `README.md`)
- Paper pipeline → Paper contracts: returns paper data (evidence: `README.md`)
- Gemini service → Extraction contracts: uses typed output (evidence: `app/services/gemini.py`)
- Example retrieval → Example contracts: returns matches (evidence: `app/services/example_retrieval.py`)
- Relational models → PostgreSQL / pgvector: defines schema (evidence: `README.md`)