How adrian-sit/paperprobe works
PaperProbe ingests research papers from PDFs or OpenReview, preserves revision and source provenance, and produces structured paper context and discussion questions. The main workflow runs through FastAPI routes into ingestion and parsing, persists versioned records, then generates questions through Gemini or a bounded agent workflow; readers can also manage a separate example library. The graph includes storage, retrieval, and request-scoped progress capabilities documented in the README. Some source excerpts are partial, so unsampled implementation links are shown only when the README documents them.
Main entry points: paper_pipeline.py Paper pipeline · agent_graph.py Agent workflow · versioned_papers.py Paper routes · example_retrieval.py Example retrieval
Components 27
API and UI
- Browser interface
app/web/app.js - Paper routes
app/api/routes/versioned_papers.py - Example routes
app/api/routes/examples.py - Progress SSE
app/api/progress.py - Paper contracts
app/schemas/papers.py - Extraction contracts
app/schemas/extraction.py - Example contracts
app/schemas/examples.py
Paper ingestion
- Paper pipeline
app/services/paper_pipeline.py - OpenReview adapter
app/services/openreview.py - Raw source archive
app/services/openreview_archive.py - PDF parsing
app/services/pdf.py - Section normalization
app/services/paper_sections.py
Question intelligence
- Agent workflow
app/services/agent_graph.py - Agent tools
app/services/agent_tools.py - Critique policy
app/services/agent_policy.py - Gemini service
app/services/gemini.py - Example library
app/services/example_library.py - Example upload
app/services/example_upload.py - Example retrieval
app/services/example_retrieval.py
Persistence and retrieval
- Runtime settings
app/core/config.py - Async DB sessions
app/db/session.py - Relational models
app/db/models.py - PostgreSQL / pgvector
- MongoDB archive
Other components
- Reader / client
- OpenReview service
- Gemini API
Connections 35
- Reader / client → Browser interface: uses
README.md - Browser interface → Paper routes: calls REST
README.md - Reader / client → Paper routes: calls REST
README.md - Paper routes → Paper pipeline: invokes ingestion
app/api/routes/versioned_papers.py - Paper routes → Agent workflow: starts workflow
app/api/routes/versioned_papers.py - Paper routes → Progress SSE: streams activity
app/api/routes/versioned_papers.py - Progress SSE → Reader / client: streams results
README.md - Example routes → Example library: manages examples
app/api/routes/examples.py - Example routes → Example upload: uploads pairs
app/api/routes/examples.py - Paper pipeline → OpenReview adapter: fetches source
app/services/paper_pipeline.py - Paper pipeline → PDF parsing: parses PDF
README.md - PDF parsing → Section normalization: normalizes text
app/services/pdf.py - Paper pipeline → Gemini service: extracts fields
app/services/paper_pipeline.py - OpenReview adapter → OpenReview service: fetches notes
README.md - OpenReview adapter → Raw source archive: archives responses
app/services/openreview.py - Raw source archive → MongoDB archive: writes snapshots
README.md - Paper pipeline → PostgreSQL / pgvector: persists revisions
README.md - Paper pipeline → Async DB sessions: uses sessions
app/services/paper_pipeline.py - Agent workflow → Agent tools: dispatches tools
app/services/agent_graph.py - Agent workflow → Critique policy: applies policy
app/services/agent_graph.py - Agent workflow → Gemini service: critiques questions
README.md - Agent workflow → Example retrieval: retrieves examples
README.md - Agent workflow → PostgreSQL / pgvector: reads and saves
README.md - Gemini service → Gemini API: generates content
README.md - Example retrieval → Gemini API: embeds context
app/services/example_retrieval.py - Example retrieval → PostgreSQL / pgvector: ranks examples
app/services/example_retrieval.py - Example upload → PDF parsing: parses paper
app/services/example_upload.py - Example upload → Example retrieval: creates embedding
app/services/example_upload.py - Example library → PostgreSQL / pgvector: stores pairs
app/services/example_library.py - Async DB sessions → Runtime settings: loads DSN
app/db/session.py - Async DB sessions → PostgreSQL / pgvector: opens connection
README.md - Paper pipeline → Paper contracts: returns paper data
README.md - Gemini service → Extraction contracts: uses typed output
app/services/gemini.py - Example retrieval → Example contracts: returns matches
app/services/example_retrieval.py - Relational models → PostgreSQL / pgvector: defines schema
README.md