How 21mohit-dotcom/vector_database_design_for_rag_system works
PrecisionRAG is an MS MARCO passage-retrieval system backed by Qdrant. Its main workflow indexes cleaned passages with dense and BM25 vectors, then serves filtered dense, keyword, or hybrid queries through an API; hybrid results can be fused and reranked before returning passages and timings. The README also documents metadata/topic filtering, live document updates, AI answers, dashboards, and evaluation. Edges follow the shown caller relationships or explicit README workflows; some documented subsystems have only directory-level paths because their implementation was not sampled.
Main entry points: retriever.py Search pipelines · main.py FastAPI service · updates.py Live upsert and delete · build_index.py Index builder
Components 22
Corpus Indexing
- Index builder
scripts/build_index.py - Clean and deduplicate
precisionrag/data.py
Query Retrieval
- Dense and BM25 encoders
precisionrag/encoders.py - Qdrant store
precisionrag/store.py - Search pipelines
precisionrag/retriever.py - Rank fusion
precisionrag/fusion.py - Cross-encoder rerank
precisionrag/rerank.py
API and UI
- FastAPI service
api/main.py - Search and dashboards UI
web/app.js - AI cited answers
precisionrag/answer.py - Result cache
precisionrag/cache.py
Updates and Metadata
- Metadata tagging
precisionrag/tags.py - Live upsert and delete
precisionrag/updates.py - Version history
precisionrag/changelog.py
Evaluation and Insights
- Results dashboard data
precisionrag/dashboard.py - Retrieval evaluation
scripts/eval_ir.py - Ranking metrics
precisionrag/metrics.py - Benchmark results
precisionrag/results.py
Other components
- Search user
- MS MARCO passages
- Qdrant service
- Groq judge service
Connections 29
- MS MARCO passages → Index builder: provides passages
README.md - Index builder → Clean and deduplicate: builds corpus
scripts/build_index.py - Index builder → Dense and BM25 encoders: encodes passages
scripts/build_index.py - Index builder → Qdrant store: writes index
scripts/build_index.py - Qdrant store → Qdrant service: reads and writes
precisionrag/store.py - Search user → Search and dashboards UI: submits query
README.md - Search and dashboards UI → FastAPI service: calls API
README.md - FastAPI service → Search pipelines: dispatches search
api/main.py - Search pipelines → Dense and BM25 encoders: encodes query
precisionrag/retriever.py - Search pipelines → Qdrant store: fetches candidates
precisionrag/retriever.py - Search pipelines → Rank fusion: fuses rankings
precisionrag/retriever.py - Search pipelines → Cross-encoder rerank: reranks candidates
precisionrag/retriever.py - Search pipelines → Metadata tagging: routes by topic
precisionrag/retriever.py - Search pipelines → Search user: returns passages
README.md - FastAPI service → Result cache: checks and fills
api/main.py - FastAPI service → Live upsert and delete: routes updates
api/main.py - Live upsert and delete → Clean and deduplicate: prepares documents
precisionrag/updates.py - Live upsert and delete → Dense and BM25 encoders: embeds documents
precisionrag/updates.py - Live upsert and delete → Qdrant store: upserts or deletes
precisionrag/updates.py - Live upsert and delete → Metadata tagging: assigns metadata
precisionrag/updates.py - Live upsert and delete → Version history: records versions
precisionrag/updates.py - FastAPI service → AI cited answers: generates answer
api/main.py - AI cited answers → Groq judge service: requests generation
README.md - FastAPI service → Results dashboard data: serves dashboard data
api/main.py - Search and dashboards UI → Results dashboard data: displays measurements
README.md - Retrieval evaluation → Search pipelines: evaluates modes
scripts/eval_ir.py - Retrieval evaluation → Ranking metrics: scores rankings
scripts/eval_ir.py - Retrieval evaluation → Benchmark results: saves measurements
scripts/eval_ir.py - Results dashboard data → Benchmark results: reads measurements
precisionrag/dashboard.py