How sourangshupal/betterdb-yt-collab works
This repository is a FastAPI RAG demo that creates real Redis data for BetterDB observability. The user workflow uploads a PDF for extraction, chunking, embedding and storage, then submits queries that are rate-limited, embedded, checked against a semantic cache, matched to stored chunks, answered by an LLM, and recorded in cache and session keys. Stats and health endpoints expose Redis key counts and configuration. BetterDB monitoring and MCP workflows are README-documented external capabilities; their implementation is not in this repository.
Main entry points: pipeline.py Query orchestration · main.py FastAPI endpoints · config.py Client configuration
Components 17
API
- FastAPI endpoints
rag/main.py - Health and stats
rag/main.py
RAG pipeline
- PDF ingestion
rag/pipeline.py - Chunking and embeddings
rag/pipeline.py - Query orchestration
rag/pipeline.py - Rate limiting
rag/pipeline.py - Semantic cache lookup
rag/pipeline.py - Document retrieval
rag/pipeline.py - Answer generation
rag/pipeline.py - Cache and session writes
rag/pipeline.py
Redis and AI services
- Redis / Valkey
- OpenAI API
- Client configuration
rag/config.py
Observability
- BetterDB monitoring
- BetterDB MCP
Other components
- API user
- Claude Code user
Connections 24
- API user → FastAPI endpoints: uploads or queries
README.md - FastAPI endpoints → PDF ingestion: routes PDF
rag/main.py - FastAPI endpoints → Query orchestration: routes query
rag/main.py - FastAPI endpoints → Health and stats: serves status
rag/main.py - PDF ingestion → Chunking and embeddings: extracts and embeds
rag/pipeline.py - Chunking and embeddings → OpenAI API: requests embeddings
rag/pipeline.py - PDF ingestion → Redis / Valkey: writes document hashes
rag/pipeline.py - Query orchestration → Rate limiting: checks limits
rag/pipeline.py - Query orchestration → OpenAI API: embeds query
rag/pipeline.py - Query orchestration → Semantic cache lookup: checks cache
rag/pipeline.py - Semantic cache lookup → Redis / Valkey: scans cache keys
rag/pipeline.py - Query orchestration → Document retrieval: retrieves on miss
rag/pipeline.py - Document retrieval → Redis / Valkey: scans document keys
rag/pipeline.py - Query orchestration → Answer generation: requests answer
rag/pipeline.py - Answer generation → OpenAI API: calls chat completion
rag/pipeline.py - Query orchestration → Cache and session writes: stores answer and session
rag/pipeline.py - Cache and session writes → Redis / Valkey: writes cache and session
rag/pipeline.py - Rate limiting → Redis / Valkey: increments counters
rag/pipeline.py - FastAPI endpoints → Redis / Valkey: scans health and stats
rag/main.py - Client configuration → Redis / Valkey: creates client
rag/config.py - Client configuration → OpenAI API: creates client
rag/config.py - Redis / Valkey → BetterDB monitoring: observed by agent
README.md - BetterDB monitoring → BetterDB MCP: serves monitoring data
README.md - Claude Code user → BetterDB MCP: asks questions
README.md