How iamnikhil-adev/clinsearch works
ClinSearch is a biomedical literature search and evidence-grounded RAG platform. Its main user workflow accepts a query through the CLI, REST API, or dashboard; embeds it, runs hybrid retrieval and reranking, then resolves and packages passages for citation-checked answer generation or deterministic abstention. A separate corpus workflow ingests PubMed records, persists canonical documents, chunks them, and builds a searchable index. The README documents evaluation and offline demo seeding as additional capabilities. Some subsystem links reflect that documented architecture; unsampled implementation details are not inferred.
Main entry points: generation/pipeline.py RAG Pipeline · retrieval/pipeline.py Retrieval Pipeline · app_service.py Application Service · service.py Embedding Service
Components 29
User Interfaces
- CLI
src/clinsearch/cli.py - REST API
src/clinsearch/api/routes.py - Dashboard
src/clinsearch/ui/static/app.js - Application Service
src/clinsearch/service/app_service.py
Corpus Ingestion
- PubMed Ingestion
src/clinsearch/data/pubmed/pipeline.py - Raw XML Store
src/clinsearch/data/pubmed/raw_store.py - Corpus Ingestion
src/clinsearch/data/corpus/ingest.py - Document Corpus
src/clinsearch/data/corpus/repository.py - Demo Seed
src/clinsearch/demo/seed.py
Index Construction
- Biomedical Chunking
src/clinsearch/data/chunking/chunker.py - Embedding Service
src/clinsearch/data/embeddings/service.py - MedCPT Encoder
src/clinsearch/data/embeddings/medcpt.py - Hybrid Index Builder
src/clinsearch/data/indexing/service.py - Persistent Index
src/clinsearch/data/indexing/persistence.py
Retrieval
- Retrieval Pipeline
src/clinsearch/retrieval/pipeline.py - Dense Search
src/clinsearch/data/indexing/dense.py - BM25 Search
src/clinsearch/data/indexing/lexical.py - Rank Fusion
src/clinsearch/retrieval/fusion.py - MedCPT Reranker
src/clinsearch/reranking/medcpt_reranker.py
Grounded Answers
- Chunk Resolver
src/clinsearch/retrieval/persistent_resolver.py - RAG Pipeline
src/clinsearch/generation/pipeline.py - Evidence Packer
src/clinsearch/generation/evidence.py - Prompt Builder
src/clinsearch/generation/prompt.py - Generation Provider
src/clinsearch/generation/hf_provider.py - Output Parser
src/clinsearch/generation/parser.py - Citation Validator
src/clinsearch/generation/citation.py - RAG Evaluation
src/clinsearch/evaluation/rag_evaluator.py
Other components
- Researcher
- PubMed API
Connections 33
- Researcher → CLI: runs commands
- Researcher → Dashboard: uses dashboard
- Researcher → REST API: sends requests
- Dashboard → REST API: calls REST
- CLI → Application Service: invokes
- REST API → Application Service: dispatches
- PubMed Ingestion → PubMed API: fetches records
- PubMed Ingestion → Raw XML Store: archives XML
- Corpus Ingestion → PubMed Ingestion: runs ingestion
- Corpus Ingestion → Document Corpus: adds documents
- Document Corpus → Biomedical Chunking: supplies snapshot
- Biomedical Chunking → Embedding Service: provides chunks
- Embedding Service → MedCPT Encoder: requests embeddings
- Biomedical Chunking → Hybrid Index Builder: provides chunks
- Embedding Service → Hybrid Index Builder: provides vectors
- Hybrid Index Builder → Persistent Index: persists indexes
- Application Service → Embedding Service: embeds query
- Application Service → Retrieval Pipeline: runs retrieval
- Retrieval Pipeline → Dense Search: searches vectors
- Retrieval Pipeline → BM25 Search: searches terms
- Retrieval Pipeline → Rank Fusion: fuses candidates
- Retrieval Pipeline → MedCPT Reranker: reranks candidates
- Application Service → RAG Pipeline: runs answering
- RAG Pipeline → Retrieval Pipeline: retrieves passages
- RAG Pipeline → Chunk Resolver: resolves chunks
- RAG Pipeline → Evidence Packer: packs evidence
- RAG Pipeline → Prompt Builder: builds prompt
- RAG Pipeline → Generation Provider: generates answer
- RAG Pipeline → Output Parser: parses output
- RAG Pipeline → Citation Validator: validates claims
- RAG Evaluation → Application Service: evaluates answers
- Demo Seed → Document Corpus: seeds corpus
- Demo Seed → Persistent Index: seeds index