# johnjusvin/vision-rag architecture

> Architecture diagram of the GitHub repository johnjusvin/vision-rag, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-09-30). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/johnjusvin/vision-rag
- Repository: https://github.com/johnjusvin/vision-rag

## Overview

vision-rag is a Python library for video question answering. A user chunks a video, optionally transcribing audio and extracting frames, embeds the chunks, indexes them, retrieves relevant moments for a question, and generates an answer with source chunks. The README documents interchangeable ASR, embedding, storage, and generation providers. The graph follows the documented workflow; the metadata loader is shown as a separate capability because the excerpts do not establish that Chunker invokes it. Provider internals are not expanded.

Main entry points: [`vision_rag/video_chunker.py`](https://github.com/johnjusvin/vision-rag/blob/main/vision_rag/video_chunker.py) (Chunking and ASR), [`vision_rag/embedding.py`](https://github.com/johnjusvin/vision-rag/blob/main/vision_rag/embedding.py) (Embedding builder), [`vision_rag/retriever.py`](https://github.com/johnjusvin/vision-rag/blob/main/vision_rag/retriever.py) (Multimodal retriever), [`vision_rag/generator.py`](https://github.com/johnjusvin/vision-rag/blob/main/vision_rag/generator.py) (Answer generator)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_ingest["Video preparation"]
  node_loader["Video metadata loader<br/>[video_ingestion.py]"]
  node_chunker["Chunking and ASR<br/>[video_chunker.py]"]
  node_asr["ASR providers<br/>[video_chunker.py]"]
end

subgraph group_vectors["Embedding and search"]
  node_embed_builder["Embedding builder<br/>[embedding.py]"]
  node_embed_providers["Embedding providers<br/>[embedding.py]"]
  node_vector_store[("FAISS and Chroma<br/>[vectorstores.py]")]
  node_retriever["Multimodal retriever<br/>[retriever.py]"]
end

subgraph group_answer["Answer generation"]
  node_generator["Answer generator<br/>[generator.py]"]
  node_llm_providers["LLM providers<br/>[generator.py]"]
  node_answer_result["Answer and sources<br/>[generator.py]"]
end

subgraph group_runtime["Shared runtime"]
  node_device["Device resolver<br/>[_device.py]"]
end

node_user(("Python user"))

node_user -->|"loads video metadata"| node_loader
node_user -->|"chunks video"| node_chunker
node_chunker -->|"transcribes audio"| node_asr
node_user -->|"embeds chunks"| node_embed_builder
node_embed_builder -->|"creates vectors"| node_embed_providers
node_embed_builder -->|"resolves device"| node_device
node_user -->|"indexes chunks"| node_vector_store
node_user -->|"queries moments"| node_retriever
node_retriever -->|"embeds query"| node_embed_providers
node_retriever -->|"searches indexes"| node_vector_store
node_user -->|"requests answer"| node_generator
node_generator -->|"generates response"| node_llm_providers
node_generator -->|"returns answer and sources"| node_answer_result
node_chunker -->|"resolves device"| node_device

click node_loader "https://github.com/johnjusvin/vision-rag/blob/main/vision_rag/video_ingestion.py"
click node_chunker "https://github.com/johnjusvin/vision-rag/blob/main/vision_rag/video_chunker.py"
click node_asr "https://github.com/johnjusvin/vision-rag/blob/main/vision_rag/video_chunker.py"
click node_embed_builder "https://github.com/johnjusvin/vision-rag/blob/main/vision_rag/embedding.py"
click node_embed_providers "https://github.com/johnjusvin/vision-rag/blob/main/vision_rag/embedding.py"
click node_vector_store "https://github.com/johnjusvin/vision-rag/blob/main/vision_rag/vectorstores.py"
click node_retriever "https://github.com/johnjusvin/vision-rag/blob/main/vision_rag/retriever.py"
click node_generator "https://github.com/johnjusvin/vision-rag/blob/main/vision_rag/generator.py"
click node_llm_providers "https://github.com/johnjusvin/vision-rag/blob/main/vision_rag/generator.py"
click node_device "https://github.com/johnjusvin/vision-rag/blob/main/vision_rag/_device.py"
click node_answer_result "https://github.com/johnjusvin/vision-rag/blob/main/vision_rag/generator.py"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_loader,node_chunker,node_asr,node_user toneBlue
class node_embed_builder,node_embed_providers,node_vector_store,node_retriever toneAmber
class node_generator,node_llm_providers,node_answer_result toneMint
class node_device toneRose
```

## Components

### Video preparation

- [Video metadata loader](https://github.com/johnjusvin/vision-rag/blob/main/vision_rag/video_ingestion.py): `vision_rag/video_ingestion.py`
- [Chunking and ASR](https://github.com/johnjusvin/vision-rag/blob/main/vision_rag/video_chunker.py): `vision_rag/video_chunker.py`
- [ASR providers](https://github.com/johnjusvin/vision-rag/blob/main/vision_rag/video_chunker.py): `vision_rag/video_chunker.py`

### Embedding and search

- [Embedding builder](https://github.com/johnjusvin/vision-rag/blob/main/vision_rag/embedding.py): `vision_rag/embedding.py`
- [Embedding providers](https://github.com/johnjusvin/vision-rag/blob/main/vision_rag/embedding.py): `vision_rag/embedding.py`
- [FAISS and Chroma](https://github.com/johnjusvin/vision-rag/blob/main/vision_rag/vectorstores.py): `vision_rag/vectorstores.py`
- [Multimodal retriever](https://github.com/johnjusvin/vision-rag/blob/main/vision_rag/retriever.py): `vision_rag/retriever.py`

### Answer generation

- [Answer generator](https://github.com/johnjusvin/vision-rag/blob/main/vision_rag/generator.py): `vision_rag/generator.py`
- [LLM providers](https://github.com/johnjusvin/vision-rag/blob/main/vision_rag/generator.py): `vision_rag/generator.py`
- [Answer and sources](https://github.com/johnjusvin/vision-rag/blob/main/vision_rag/generator.py): `vision_rag/generator.py`

### Shared runtime

- [Device resolver](https://github.com/johnjusvin/vision-rag/blob/main/vision_rag/_device.py): `vision_rag/_device.py`

### Other components

- **Python user**

## Connections

- Python user → Video metadata loader: loads video metadata (evidence: `vision_rag/video_ingestion.py`)
- Python user → Chunking and ASR: chunks video (evidence: `README.md`)
- Chunking and ASR → ASR providers: transcribes audio (evidence: `README.md`)
- Python user → Embedding builder: embeds chunks (evidence: `README.md`)
- Embedding builder → Embedding providers: creates vectors (evidence: `vision_rag/embedding.py`)
- Embedding builder → Device resolver: resolves device (evidence: `vision_rag/embedding.py`)
- Python user → FAISS and Chroma: indexes chunks (evidence: `README.md`)
- Python user → Multimodal retriever: queries moments (evidence: `README.md`)
- Multimodal retriever → Embedding providers: embeds query (evidence: `vision_rag/retriever.py`)
- Multimodal retriever → FAISS and Chroma: searches indexes (evidence: `vision_rag/retriever.py`)
- Python user → Answer generator: requests answer (evidence: `README.md`)
- Answer generator → LLM providers: generates response (evidence: `vision_rag/generator.py`)
- Answer generator → Answer and sources: returns answer and sources (evidence: `vision_rag/generator.py`)
- Chunking and ASR → Device resolver: resolves device (evidence: `vision_rag/video_chunker.py`)
