How devashish2006/video_rag works
Project type and purpose
• This is a Python‐based Retrieval-Augmented Generation (RAG) system focused on video question answering.
• It ingests a video, transcribes speech via Whisper, splits it into timestamped chunks, embeds those chunks with Gemini, stores/query them in Qdrant, and finally uses a LangChain + Gemini pipeline to answer user questions with context and timestamps.
File structure
• Top level:
• app.py (or video_rag.py): the orchestrator script that ties everything together
• docker-compose.yml: defines a local Qdrant service
• requirements.txt: Python dependencies (whisper, langchain, google-generativeai, qdrant-client)
• readme.md: project description and usage
• No separate “frontend” folder suggests a CLI or single-script interface, not a web GUI.
• The monolithic Python script internally implements multiple functional stages (transcription, chunking, embedding, storage, retrieval, QA).
Key insights from README
• External services/APIs: Whisper (OpenAI model), Google Generative AI (Gemini), LangChain wrapper, Qdrant vector DB.
• Deployment: Qdrant is run via Docker locally; the rest is pure Python.
Main components
a. User (CLI)
b. Orchestrator (app.py)
• Ingest video file
• Call Whisper transcription service
• Run chunking logic (+ timestamps)
• Call Gemini embedding endpoint
• Upload embeddings to Qdrant
• Accept user question, run LangChain RAG using Gemini QA
• Return answer + relevant timestamps to user
c. External LLM services
• Whisper (speech-to-text)
• Google Generative AI (embedding & QA via Gemini models)
d. Vector Database
• Qdrant container (runs in Docker)
Relationships and data flow
1) User → starts app.py with a video file
2) app.py → Whisper API → raw transcript
3) app.py → chunker → segments (with timestamp metadata)
4) app.py → Gemini embedding API → embedding vectors
5) app.py → Qdrant client → store embeddings + metadata
6) User asks question → app.py → Qdrant client → relevant embeddings
7) app.py → Gemini QA API (via LangChain) → answer
8) app.py → CLI output (answer + timestamps)
Architectural patterns and design principles
• Pipeline pattern: linear stages (transcription → chunking → embedding → storage → retrieval → QA)
• Separation of concerns: each stage (Whisper, chunking, embedding, vector DB, QA) is a distinct module/step
• Single deployment unit for Python code; Qdrant isolated in Docker
Detail level
• External dependencies (Docker Qdrant, Whisper, Gemini).
• Don’t just group everything under “backend”—explicitly separate transcription, embedding, storage, retrieval, and QA.