How kishormorol/researchscope works
ResearchScope combines a daily paper-ingestion and enrichment pipeline, a FastAPI-backed searchable catalogue, and a lightweight local CLI. The primary workflow gathers papers from scholarly sources, enriches and analyzes them, then publishes data to the site and storage backends for researchers to search and explore. The README also documents venue recommenders, deadlines, author intelligence, and paper chat. Some branches are shown from README documentation because their source was not sampled; unsampled implementation details and wiring are not inferred.
Main entry points: pipeline.py Pipeline runner · main.py REST API · cli.py ResearchScope CLI · database.py API database
Components 30
Ingestion pipeline
- Pipeline runner
src/pipeline.py - Source connectors
src/connectors - Paper schema
src/normalization/schema.py - Deduplication
src/dedup/deduplicator.py - Topic tagging
src/tagging/tagger.py - Reading difficulty
src/difficulty/assessor.py
Research intelligence
- Paper scoring
src/scoring/scorer.py - Paper enrichment
src/content/generator.py - Topic clustering
src/clustering/clusterer.py - Research gaps
src/gaps/gap_extractor.py - Author profiles
src/aggregation/aggregator.py - Venue recommenders
src/sitegen
Publishing and storage
- Site generation
src/sitegen/generator.py - Railway catalogue
src/storage/railway_store.py - Hugging Face dataset
src/storage/hf_dataset.py
API and paper chat
- REST API
backend/app/main.py - Paper and search routes
backend/app/routers - Accounts and favourites
backend/app/routers - Chat and document routes
backend/app/routers - PDF preparation
backend/app/services/document_service.py - Paper chat
backend/app/services/chat_service.py - Evidence retrieval
backend/app/services/retrieval_service.py - Model providers
backend/app/services/provider_service.py - API database
backend/app/database.py
Local CLI
- ResearchScope CLI
researchscope/cli.py - CLI collectors
researchscope/collectors - Local paper store
researchscope/storage/store.py - Saved-paper analysis
researchscope/analysis
Other components
- Researcher
- Scholarly sources
Connections 33
- Scholarly sources → Source connectors: provides papers
README.md - Pipeline runner → Source connectors: fetches papers
src/pipeline.py - Pipeline runner → Deduplication: deduplicates
README.md - Pipeline runner → Topic tagging: tags papers
README.md - Pipeline runner → Reading difficulty: assesses difficulty
README.md - Pipeline runner → Paper scoring: scores papers
src/pipeline.py - Pipeline runner → Paper enrichment: enriches papers
src/pipeline.py - Pipeline runner → Topic clustering: groups topics
README.md - Pipeline runner → Research gaps: extracts gaps
README.md - Pipeline runner → Author profiles: builds profiles
README.md - Pipeline runner → Site generation: generates site data
README.md - Pipeline runner → Railway catalogue: syncs papers
README.md - Pipeline runner → Hugging Face dataset: syncs dataset
README.md - Researcher → REST API: uses
README.md - REST API → Paper and search routes: registers routes
backend/app/main.py - REST API → Accounts and favourites: registers routes
backend/app/main.py - REST API → Chat and document routes: registers routes
backend/app/main.py - Paper and search routes → API database: queries catalogue
backend/app/routers/papers.py - Accounts and favourites → API database: reads and writes
backend/app/routers/favourites.py - Chat and document routes → PDF preparation: prepares documents
backend/app/routers/paper_documents.py - Chat and document routes → Paper chat: handles chat
backend/app/routers/chat.py - Paper chat → Evidence retrieval: retrieves evidence
backend/app/services/chat_service.py - Paper chat → Model providers: requests responses
backend/app/services/chat_service.py - PDF preparation → API database: stores document data
backend/app/services/document_service.py - Evidence retrieval → API database: searches chunks
backend/app/services/retrieval_service.py - Researcher → Venue recommenders: gets venue matches
README.md - Researcher → Research gaps: explores gaps
README.md - Researcher → Author profiles: explores researchers
README.md - Researcher → ResearchScope CLI: runs commands
README.md - ResearchScope CLI → CLI collectors: searches sources
researchscope/cli.py - ResearchScope CLI → Local paper store: saves papers
researchscope/cli.py - ResearchScope CLI → Saved-paper analysis: analyzes saved papers
README.md - CLI collectors → Scholarly sources: queries
README.md