How hungphung25121981-commits/video-rag-system works
This is a local-first, multimodal video-RAG backend. The principal workflow starts with a video directory, extracts audio, visual/OCR information and chunks, builds vector and temporal/spatial graph indexes, then answers natural-language queries with timestamped evidence. A CLI coordinates five documented phases; Phase 0 also provides hashing, resumable tracking and rollback, while Phase 4 evaluates answers and tunes retrieval weights. The graph follows documented phase orchestration and shown storage access; unsampled implementation details are not inferred.
Main entry points: cli_pipeline.py Pipeline CLI · retriever.py Tri-search retriever · graph_router.py Graph hop router · qdrant_manager.py Vector index manager
Components 19
Foundation & CLI
- Pipeline CLI
backend/cli_pipeline.py - Settings loader
backend/src/common/config_loader.py - Video hashing
backend/src/phase0_ops/hashing.py - Resume & rollback
backend/src/phase0_ops/dual_tracker.py - System checks
backend/src/phase0_ops/system_check.py
Video Extraction
- Video & keyframes
backend/src/phase1_extract/video_processor.py - Audio pipeline
backend/src/phase1_extract/audio_pipeline.py - OCR text processing
backend/src/phase1_extract/text_processor.py - Hybrid chunking
backend/src/phase1_extract/hybrid_chunking.py
Index Construction
- Multimodal embeddings
backend/src/phase2_build/embedding.py - Vector index manager
backend/src/phase2_build/qdrant_manager.py - Knowledge graph builder
backend/src/phase2_build/kuzu_builder.py
Query & Answers
- Tri-search retriever
backend/src/phase3_search/retriever.py - Graph hop router
backend/src/phase3_search/graph_router.py - Candidate reranker
backend/src/phase3_search/reranker.py - VLM answer generation
backend/src/phase3_search/vlm_generator.py
Evaluation & Tuning
- Answer evaluation
backend/src/phase4_eval/metric_scorer.py - Retrieval tuning
backend/src/phase4_eval/hyper_tuner.py
Other components
- Video user
Connections 20
- Video user → Pipeline CLI: runs queries
README.md - Pipeline CLI → System checks: checks system
backend/cli_pipeline.py - Pipeline CLI → Video hashing: hashes videos
backend/cli_pipeline.py - Pipeline CLI → Resume & rollback: tracks progress
backend/cli_pipeline.py - Pipeline CLI → Video & keyframes: extracts video
backend/cli_pipeline.py - Pipeline CLI → Audio pipeline: processes audio
README.md - Pipeline CLI → OCR text processing: processes text
backend/cli_pipeline.py - Pipeline CLI → Hybrid chunking: chunks content
backend/cli_pipeline.py - Pipeline CLI → Multimodal embeddings: builds vectors
backend/cli_pipeline.py - Pipeline CLI → Vector index manager: writes vector index
README.md - Pipeline CLI → Knowledge graph builder: builds graph index
README.md - Pipeline CLI → Tri-search retriever: runs tri-search
backend/cli_pipeline.py - Pipeline CLI → Graph hop router: routes graph hops
README.md - Pipeline CLI → Candidate reranker: reranks candidates
README.md - Pipeline CLI → VLM answer generation: generates answers
README.md - Pipeline CLI → Answer evaluation: evaluates answers
README.md - Pipeline CLI → Retrieval tuning: tunes retrieval
backend/cli_pipeline.py - Pipeline CLI → Settings loader: loads settings
backend/src/common/config_loader.py - Tri-search retriever → Vector index manager: queries Qdrant
backend/src/phase3_search/retriever.py - Graph hop router → Knowledge graph builder: queries Kuzu graph
backend/src/phase3_search/graph_router.py