How johnjusvin/vision-rag works
vision-rag is a Python library for video question answering. A user chunks a video, optionally transcribing audio and extracting frames, embeds the chunks, indexes them, retrieves relevant moments for a question, and generates an answer with source chunks. The README documents interchangeable ASR, embedding, storage, and generation providers. The graph follows the documented workflow; the metadata loader is shown as a separate capability because the excerpts do not establish that Chunker invokes it. Provider internals are not expanded.
Main entry points: video_chunker.py Chunking and ASR · embedding.py Embedding builder · retriever.py Multimodal retriever · generator.py Answer generator
Components 12
Video preparation
- Video metadata loader
vision_rag/video_ingestion.py - Chunking and ASR
vision_rag/video_chunker.py - ASR providers
vision_rag/video_chunker.py
Embedding and search
- Embedding builder
vision_rag/embedding.py - Embedding providers
vision_rag/embedding.py - FAISS and Chroma
vision_rag/vectorstores.py - Multimodal retriever
vision_rag/retriever.py
Answer generation
- Answer generator
vision_rag/generator.py - LLM providers
vision_rag/generator.py - Answer and sources
vision_rag/generator.py
Shared runtime
- Device resolver
vision_rag/_device.py
Other components
- Python user
Connections 14
- Python user → Video metadata loader: loads video metadata
vision_rag/video_ingestion.py - Python user → Chunking and ASR: chunks video
README.md - Chunking and ASR → ASR providers: transcribes audio
README.md - Python user → Embedding builder: embeds chunks
README.md - Embedding builder → Embedding providers: creates vectors
vision_rag/embedding.py - Embedding builder → Device resolver: resolves device
vision_rag/embedding.py - Python user → FAISS and Chroma: indexes chunks
README.md - Python user → Multimodal retriever: queries moments
README.md - Multimodal retriever → Embedding providers: embeds query
vision_rag/retriever.py - Multimodal retriever → FAISS and Chroma: searches indexes
vision_rag/retriever.py - Python user → Answer generator: requests answer
README.md - Answer generator → LLM providers: generates response
vision_rag/generator.py - Answer generator → Answer and sources: returns answer and sources
vision_rag/generator.py - Chunking and ASR → Device resolver: resolves device
vision_rag/video_chunker.py