How soummo19/prismsearch works
This is a near-real-time, multi-source retrieval and grounded RAG system. Its principal workflow captures PostgreSQL changes through Debezium and Redpanda, normalizes them into canonical documents, indexes text and vectors, then serves tenant-scoped hybrid search and cited answers. The graph also shows the ML service, semantic and response caching, and storage layers. Some internal query and indexing calls are omitted from the excerpts; documented relationships are cited to the README, and uncertain links are not asserted.
Main entry points: query/main.go Query API · indexer/main.go Indexer service · normalizer/main.go Normalizer service · rag.go Cited RAG
Components 25
Change ingestion
- Redpanda
- Normalizer service
cmd/normalizer/main.go - CDC normalizer
internal/normalizer/normalizer.go - Canonical document
internal/contract/doc.go - Fast / bulk topics
- Indexer service
cmd/indexer/main.go - Version guard
internal/idempotency/guard.go
Retrieval and RAG
- Query API
cmd/query/main.go - Hybrid fusion
internal/retrieve/fusion.go - Cited RAG
internal/rag/rag.go - Faithfulness guard
internal/rag/faithfulness.go - Semantic cache
internal/semcache/semcache.go - Response cache
internal/cache/cache.go - Cited answer
ML inference
- ML client
internal/mlclient/mlclient.go - ML HTTP service
mlservice/app.py - Embedding backend
mlservice/embedder.py - Reranker backend
mlservice/reranker.py
Stores and telemetry
- OpenSearch BM25
internal/opensearch/opensearch.go - Qdrant vectors
internal/qdrant/qdrant.go - Kafka client
internal/kafka/kafka.go - Metrics
internal/obs/metrics.go
Other components
- PostgreSQL source
- Debezium CDC
- User
Connections 26
- PostgreSQL source → Debezium CDC: streams changes
README.md - Debezium CDC → Redpanda: publishes CDC
README.md - Redpanda → Normalizer service: feeds events
cmd/normalizer/main.go - Normalizer service → CDC normalizer: parses events
cmd/normalizer/main.go - CDC normalizer → Canonical document: creates documents
internal/normalizer/normalizer.go - Normalizer service → Fast / bulk topics: routes by tier
cmd/normalizer/main.go - Fast / bulk topics → Indexer service: feeds lanes
README.md - Indexer service → Version guard: checks versions
README.md - Indexer service → ML client: requests embeddings
cmd/indexer/main.go - Indexer service → OpenSearch BM25: writes documents
cmd/indexer/main.go - Indexer service → Qdrant vectors: writes vectors
cmd/indexer/main.go - User → Query API: submits search
README.md - Query API → OpenSearch BM25: runs BM25
README.md - Query API → Qdrant vectors: runs vector search
README.md - Query API → Hybrid fusion: combines rankings
README.md - Query API → ML client: requests reranking
cmd/query/main.go - Query API → Semantic cache: checks semantic match
cmd/query/main.go - Query API → Response cache: checks cached response
cmd/query/main.go - Query API → Cited RAG: generates answer
README.md - Cited RAG → Faithfulness guard: checks grounding
README.md - Cited RAG → Cited answer: returns citations
README.md - Cited answer → User: returns answer
README.md - ML client → ML HTTP service: calls HTTP endpoints
internal/mlclient/mlclient.go - ML HTTP service → Embedding backend: embeds texts
mlservice/app.py - ML HTTP service → Reranker backend: scores documents
mlservice/app.py - Indexer service → Metrics: records freshness
README.md