# hungphung25121981-commits/video-rag-system architecture

> Architecture diagram of the GitHub repository hungphung25121981-commits/video-rag-system, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-09-30). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/hungphung25121981-commits/video-rag-system
- Repository: https://github.com/hungphung25121981-commits/video-rag-system

## Overview

This is a local-first, multimodal video-RAG backend. The principal workflow starts with a video directory, extracts audio, visual/OCR information and chunks, builds vector and temporal/spatial graph indexes, then answers natural-language queries with timestamped evidence. A CLI coordinates five documented phases; Phase 0 also provides hashing, resumable tracking and rollback, while Phase 4 evaluates answers and tunes retrieval weights. The graph follows documented phase orchestration and shown storage access; unsampled implementation details are not inferred.

Main entry points: [`backend/cli_pipeline.py`](https://github.com/hungphung25121981-commits/video-rag-system/blob/main/backend/cli_pipeline.py) (Pipeline CLI), [`backend/src/phase3_search/retriever.py`](https://github.com/hungphung25121981-commits/video-rag-system/blob/main/backend/src/phase3_search/retriever.py) (Tri-search retriever), [`backend/src/phase3_search/graph_router.py`](https://github.com/hungphung25121981-commits/video-rag-system/blob/main/backend/src/phase3_search/graph_router.py) (Graph hop router), [`backend/src/phase2_build/qdrant_manager.py`](https://github.com/hungphung25121981-commits/video-rag-system/blob/main/backend/src/phase2_build/qdrant_manager.py) (Vector index manager)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_foundation["Foundation &amp; CLI"]
  node_cli["Pipeline CLI<br/>[cli_pipeline.py]"]
  node_config["Settings loader<br/>[config_loader.py]"]
  node_hash["Video hashing<br/>[hashing.py]"]
  node_tracker[("Resume &amp; rollback<br/>[dual_tracker.py]")]
  node_systemcheck["System checks<br/>[system_check.py]"]
end

subgraph group_extract["Video Extraction"]
  node_video["Video &amp; keyframes<br/>[video_processor.py]"]
  node_audio["Audio pipeline<br/>[audio_pipeline.py]"]
  node_text["OCR text processing<br/>[text_processor.py]"]
  node_chunking["Hybrid chunking<br/>[hybrid_chunking.py]"]
end

subgraph group_build["Index Construction"]
  node_embedding["Multimodal embeddings<br/>[embedding.py]"]
  node_qdrantmanager[("Vector index manager<br/>[qdrant_manager.py]")]
  node_kuzubuilder[("Knowledge graph builder<br/>[kuzu_builder.py]")]
end

subgraph group_search["Query &amp; Answers"]
  node_retriever["Tri-search retriever<br/>[retriever.py]"]
  node_router["Graph hop router<br/>[graph_router.py]"]
  node_reranker["Candidate reranker<br/>[reranker.py]"]
  node_generator["VLM answer generation<br/>[vlm_generator.py]"]
end

subgraph group_evaluation["Evaluation &amp; Tuning"]
  node_scorer["Answer evaluation<br/>[metric_scorer.py]"]
  node_tuner["Retrieval tuning<br/>[hyper_tuner.py]"]
end

node_operator(("Video user"))

node_operator -->|"runs queries"| node_cli
node_cli -->|"checks system"| node_systemcheck
node_cli -->|"hashes videos"| node_hash
node_cli -->|"tracks progress"| node_tracker
node_cli -->|"extracts video"| node_video
node_cli -->|"processes audio"| node_audio
node_cli -->|"processes text"| node_text
node_cli -->|"chunks content"| node_chunking
node_cli -->|"builds vectors"| node_embedding
node_cli -->|"writes vector index"| node_qdrantmanager
node_cli -->|"builds graph index"| node_kuzubuilder
node_cli -->|"runs tri-search"| node_retriever
node_cli -->|"routes graph hops"| node_router
node_cli -->|"reranks candidates"| node_reranker
node_cli -->|"generates answers"| node_generator
node_cli -->|"evaluates answers"| node_scorer
node_cli -->|"tunes retrieval"| node_tuner
node_cli -->|"loads settings"| node_config
node_retriever -->|"queries Qdrant"| node_qdrantmanager
node_router -->|"queries Kuzu graph"| node_kuzubuilder

click node_cli "https://github.com/hungphung25121981-commits/video-rag-system/blob/main/backend/cli_pipeline.py"
click node_config "https://github.com/hungphung25121981-commits/video-rag-system/blob/main/backend/src/common/config_loader.py"
click node_hash "https://github.com/hungphung25121981-commits/video-rag-system/blob/main/backend/src/phase0_ops/hashing.py"
click node_tracker "https://github.com/hungphung25121981-commits/video-rag-system/blob/main/backend/src/phase0_ops/dual_tracker.py"
click node_systemcheck "https://github.com/hungphung25121981-commits/video-rag-system/blob/main/backend/src/phase0_ops/system_check.py"
click node_video "https://github.com/hungphung25121981-commits/video-rag-system/blob/main/backend/src/phase1_extract/video_processor.py"
click node_audio "https://github.com/hungphung25121981-commits/video-rag-system/blob/main/backend/src/phase1_extract/audio_pipeline.py"
click node_text "https://github.com/hungphung25121981-commits/video-rag-system/blob/main/backend/src/phase1_extract/text_processor.py"
click node_chunking "https://github.com/hungphung25121981-commits/video-rag-system/blob/main/backend/src/phase1_extract/hybrid_chunking.py"
click node_embedding "https://github.com/hungphung25121981-commits/video-rag-system/blob/main/backend/src/phase2_build/embedding.py"
click node_qdrantmanager "https://github.com/hungphung25121981-commits/video-rag-system/blob/main/backend/src/phase2_build/qdrant_manager.py"
click node_kuzubuilder "https://github.com/hungphung25121981-commits/video-rag-system/blob/main/backend/src/phase2_build/kuzu_builder.py"
click node_retriever "https://github.com/hungphung25121981-commits/video-rag-system/blob/main/backend/src/phase3_search/retriever.py"
click node_router "https://github.com/hungphung25121981-commits/video-rag-system/blob/main/backend/src/phase3_search/graph_router.py"
click node_reranker "https://github.com/hungphung25121981-commits/video-rag-system/blob/main/backend/src/phase3_search/reranker.py"
click node_generator "https://github.com/hungphung25121981-commits/video-rag-system/blob/main/backend/src/phase3_search/vlm_generator.py"
click node_scorer "https://github.com/hungphung25121981-commits/video-rag-system/blob/main/backend/src/phase4_eval/metric_scorer.py"
click node_tuner "https://github.com/hungphung25121981-commits/video-rag-system/blob/main/backend/src/phase4_eval/hyper_tuner.py"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_cli,node_config,node_hash,node_tracker,node_systemcheck,node_operator toneBlue
class node_video,node_audio,node_text,node_chunking toneAmber
class node_embedding,node_qdrantmanager,node_kuzubuilder toneMint
class node_retriever,node_router,node_reranker,node_generator toneRose
class node_scorer,node_tuner toneIndigo
```

## Components

### Foundation & CLI

- [Pipeline CLI](https://github.com/hungphung25121981-commits/video-rag-system/blob/main/backend/cli_pipeline.py): `backend/cli_pipeline.py`
- [Settings loader](https://github.com/hungphung25121981-commits/video-rag-system/blob/main/backend/src/common/config_loader.py): `backend/src/common/config_loader.py`
- [Video hashing](https://github.com/hungphung25121981-commits/video-rag-system/blob/main/backend/src/phase0_ops/hashing.py): `backend/src/phase0_ops/hashing.py`
- [Resume & rollback](https://github.com/hungphung25121981-commits/video-rag-system/blob/main/backend/src/phase0_ops/dual_tracker.py): `backend/src/phase0_ops/dual_tracker.py`
- [System checks](https://github.com/hungphung25121981-commits/video-rag-system/blob/main/backend/src/phase0_ops/system_check.py): `backend/src/phase0_ops/system_check.py`

### Video Extraction

- [Video & keyframes](https://github.com/hungphung25121981-commits/video-rag-system/blob/main/backend/src/phase1_extract/video_processor.py): `backend/src/phase1_extract/video_processor.py`
- [Audio pipeline](https://github.com/hungphung25121981-commits/video-rag-system/blob/main/backend/src/phase1_extract/audio_pipeline.py): `backend/src/phase1_extract/audio_pipeline.py`
- [OCR text processing](https://github.com/hungphung25121981-commits/video-rag-system/blob/main/backend/src/phase1_extract/text_processor.py): `backend/src/phase1_extract/text_processor.py`
- [Hybrid chunking](https://github.com/hungphung25121981-commits/video-rag-system/blob/main/backend/src/phase1_extract/hybrid_chunking.py): `backend/src/phase1_extract/hybrid_chunking.py`

### Index Construction

- [Multimodal embeddings](https://github.com/hungphung25121981-commits/video-rag-system/blob/main/backend/src/phase2_build/embedding.py): `backend/src/phase2_build/embedding.py`
- [Vector index manager](https://github.com/hungphung25121981-commits/video-rag-system/blob/main/backend/src/phase2_build/qdrant_manager.py): `backend/src/phase2_build/qdrant_manager.py`
- [Knowledge graph builder](https://github.com/hungphung25121981-commits/video-rag-system/blob/main/backend/src/phase2_build/kuzu_builder.py): `backend/src/phase2_build/kuzu_builder.py`

### Query & Answers

- [Tri-search retriever](https://github.com/hungphung25121981-commits/video-rag-system/blob/main/backend/src/phase3_search/retriever.py): `backend/src/phase3_search/retriever.py`
- [Graph hop router](https://github.com/hungphung25121981-commits/video-rag-system/blob/main/backend/src/phase3_search/graph_router.py): `backend/src/phase3_search/graph_router.py`
- [Candidate reranker](https://github.com/hungphung25121981-commits/video-rag-system/blob/main/backend/src/phase3_search/reranker.py): `backend/src/phase3_search/reranker.py`
- [VLM answer generation](https://github.com/hungphung25121981-commits/video-rag-system/blob/main/backend/src/phase3_search/vlm_generator.py): `backend/src/phase3_search/vlm_generator.py`

### Evaluation & Tuning

- [Answer evaluation](https://github.com/hungphung25121981-commits/video-rag-system/blob/main/backend/src/phase4_eval/metric_scorer.py): `backend/src/phase4_eval/metric_scorer.py`
- [Retrieval tuning](https://github.com/hungphung25121981-commits/video-rag-system/blob/main/backend/src/phase4_eval/hyper_tuner.py): `backend/src/phase4_eval/hyper_tuner.py`

### Other components

- **Video user**

## Connections

- Video user → Pipeline CLI: runs queries (evidence: `README.md`)
- Pipeline CLI → System checks: checks system (evidence: `backend/cli_pipeline.py`)
- Pipeline CLI → Video hashing: hashes videos (evidence: `backend/cli_pipeline.py`)
- Pipeline CLI → Resume & rollback: tracks progress (evidence: `backend/cli_pipeline.py`)
- Pipeline CLI → Video & keyframes: extracts video (evidence: `backend/cli_pipeline.py`)
- Pipeline CLI → Audio pipeline: processes audio (evidence: `README.md`)
- Pipeline CLI → OCR text processing: processes text (evidence: `backend/cli_pipeline.py`)
- Pipeline CLI → Hybrid chunking: chunks content (evidence: `backend/cli_pipeline.py`)
- Pipeline CLI → Multimodal embeddings: builds vectors (evidence: `backend/cli_pipeline.py`)
- Pipeline CLI → Vector index manager: writes vector index (evidence: `README.md`)
- Pipeline CLI → Knowledge graph builder: builds graph index (evidence: `README.md`)
- Pipeline CLI → Tri-search retriever: runs tri-search (evidence: `backend/cli_pipeline.py`)
- Pipeline CLI → Graph hop router: routes graph hops (evidence: `README.md`)
- Pipeline CLI → Candidate reranker: reranks candidates (evidence: `README.md`)
- Pipeline CLI → VLM answer generation: generates answers (evidence: `README.md`)
- Pipeline CLI → Answer evaluation: evaluates answers (evidence: `README.md`)
- Pipeline CLI → Retrieval tuning: tunes retrieval (evidence: `backend/cli_pipeline.py`)
- Pipeline CLI → Settings loader: loads settings (evidence: `backend/src/common/config_loader.py`)
- Tri-search retriever → Vector index manager: queries Qdrant (evidence: `backend/src/phase3_search/retriever.py`)
- Graph hop router → Knowledge graph builder: queries Kuzu graph (evidence: `backend/src/phase3_search/graph_router.py`)
