# vk-ant/sightrag architecture

> Architecture diagram of the GitHub repository vk-ant/sightrag, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-10-03). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/vk-ant/sightrag
- Repository: https://github.com/vk-ant/sightrag

## Overview

SightRAG is a pluggable visual retrieval system: users index images, videos, or camera input, then search by text or reference image and inspect results. Its documented processing stages include detection, optional segmentation and OCR, embedding, tracking, and storage; retrieval supports optional reranking and multimodal understanding, with visualization and timeline features. The graph includes documented capabilities whose source was not excerpted. Some optional wiring is therefore shown only where the README or sampled source supports it.

Main entry points: [`sightrag/core.py`](https://github.com/vk-ant/sightrag/blob/main/sightrag/core.py) (SightRAG), [`sightrag/indexer.py`](https://github.com/vk-ant/sightrag/blob/main/sightrag/indexer.py) (Indexer), [`sightrag/retriever.py`](https://github.com/vk-ant/sightrag/blob/main/sightrag/retriever.py) (Retriever), [`sightrag/store/sqlite_store.py`](https://github.com/vk-ant/sightrag/blob/main/sightrag/store/sqlite_store.py) (Vector and track store)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_ingest["Ingestion"]
  node_inputs["Visual inputs"]
  node_sightrag["SightRAG<br/>[core.py]"]
  node_indexer["Indexer<br/>[indexer.py]"]
  node_video["Frame extraction<br/>[video.py]"]
end

subgraph group_models["Models"]
  node_detector["Detection<br/>[base.py]"]
  node_segmentor["Segmentation<br/>[base.py]"]
  node_embedder["Embedding<br/>[base.py]"]
  node_backends["Inference backends<br/>[base.py]"]
  node_ocr["OCR<br/>[ocr.py]"]
end

subgraph group_search["Retrieval"]
  node_store[("Vector and track store<br/>[sqlite_store.py]")]
  node_retriever["Retriever<br/>[retriever.py]"]
  node_reranker["Reranking<br/>[reranker.py]"]
  node_multimodal["Multimodal understanding<br/>[multimodal.py]"]
end

subgraph group_tracking["Tracking"]
  node_tracker["Object tracking<br/>[bytetrack.py]"]
end

subgraph group_access["Interfaces"]
  node_visualizer["Results and timelines<br/>[visualizer.py]"]
  node_cli["CLI<br/>[cli.py]"]
  node_api["REST API<br/>[api.py]"]
end

node_user(("User"))

node_user -->|"uses"| node_sightrag
node_inputs -->|"indexes"| node_sightrag
node_sightrag -->|"dispatches indexing"| node_indexer
node_indexer -->|"extracts frames"| node_video
node_indexer -.->|"detects objects"| node_detector
node_indexer -.->|"segments inputs"| node_segmentor
node_indexer -.->|"embeds regions"| node_embedder
node_indexer -.->|"reads text"| node_ocr
node_indexer -.->|"tracks detections"| node_tracker
node_sightrag -->|"selects backend"| node_backends
node_indexer -.->|"writes vectors"| node_store
node_indexer -.->|"saves tracks"| node_store
node_sightrag -->|"dispatches queries"| node_retriever
node_retriever -->|"embeds query"| node_embedder
node_retriever -->|"detects reference"| node_detector
node_retriever -->|"searches vectors"| node_store
node_sightrag -.->|"reranks results"| node_reranker
node_sightrag -.->|"understands queries"| node_multimodal
node_sightrag -->|"reads track history"| node_store
node_sightrag -->|"renders results"| node_visualizer
node_user -->|"runs commands"| node_cli
node_user -.->|"sends requests"| node_api
node_cli -.->|"invokes features"| node_sightrag
node_api -.->|"serves operations"| node_sightrag

click node_sightrag "https://github.com/vk-ant/sightrag/blob/main/sightrag/core.py"
click node_indexer "https://github.com/vk-ant/sightrag/blob/main/sightrag/indexer.py"
click node_video "https://github.com/vk-ant/sightrag/blob/main/sightrag/utils/video.py"
click node_detector "https://github.com/vk-ant/sightrag/blob/main/sightrag/detectors/base.py"
click node_segmentor "https://github.com/vk-ant/sightrag/blob/main/sightrag/segmentors/base.py"
click node_embedder "https://github.com/vk-ant/sightrag/blob/main/sightrag/embedders/base.py"
click node_backends "https://github.com/vk-ant/sightrag/blob/main/sightrag/backends/base.py"
click node_ocr "https://github.com/vk-ant/sightrag/blob/main/sightrag/ocr.py"
click node_tracker "https://github.com/vk-ant/sightrag/blob/main/sightrag/trackers/bytetrack.py"
click node_store "https://github.com/vk-ant/sightrag/blob/main/sightrag/store/sqlite_store.py"
click node_retriever "https://github.com/vk-ant/sightrag/blob/main/sightrag/retriever.py"
click node_reranker "https://github.com/vk-ant/sightrag/blob/main/sightrag/reranker.py"
click node_multimodal "https://github.com/vk-ant/sightrag/blob/main/sightrag/multimodal.py"
click node_visualizer "https://github.com/vk-ant/sightrag/blob/main/sightrag/visualizer.py"
click node_cli "https://github.com/vk-ant/sightrag/blob/main/sightrag/cli.py"
click node_api "https://github.com/vk-ant/sightrag/blob/main/sightrag/api.py"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_inputs,node_sightrag,node_indexer,node_video,node_user toneBlue
class node_detector,node_segmentor,node_embedder,node_backends,node_ocr toneAmber
class node_store,node_retriever,node_reranker,node_multimodal toneMint
class node_tracker toneRose
class node_visualizer,node_cli,node_api toneIndigo
```

## Components

### Ingestion

- **Visual inputs**
- [SightRAG](https://github.com/vk-ant/sightrag/blob/main/sightrag/core.py): `sightrag/core.py`
- [Indexer](https://github.com/vk-ant/sightrag/blob/main/sightrag/indexer.py): `sightrag/indexer.py`
- [Frame extraction](https://github.com/vk-ant/sightrag/blob/main/sightrag/utils/video.py): `sightrag/utils/video.py`

### Models

- [Detection](https://github.com/vk-ant/sightrag/blob/main/sightrag/detectors/base.py): `sightrag/detectors/base.py`
- [Segmentation](https://github.com/vk-ant/sightrag/blob/main/sightrag/segmentors/base.py): `sightrag/segmentors/base.py`
- [Embedding](https://github.com/vk-ant/sightrag/blob/main/sightrag/embedders/base.py): `sightrag/embedders/base.py`
- [Inference backends](https://github.com/vk-ant/sightrag/blob/main/sightrag/backends/base.py): `sightrag/backends/base.py`
- [OCR](https://github.com/vk-ant/sightrag/blob/main/sightrag/ocr.py): `sightrag/ocr.py`

### Retrieval

- [Vector and track store](https://github.com/vk-ant/sightrag/blob/main/sightrag/store/sqlite_store.py): `sightrag/store/sqlite_store.py`
- [Retriever](https://github.com/vk-ant/sightrag/blob/main/sightrag/retriever.py): `sightrag/retriever.py`
- [Reranking](https://github.com/vk-ant/sightrag/blob/main/sightrag/reranker.py): `sightrag/reranker.py`
- [Multimodal understanding](https://github.com/vk-ant/sightrag/blob/main/sightrag/multimodal.py): `sightrag/multimodal.py`

### Tracking

- [Object tracking](https://github.com/vk-ant/sightrag/blob/main/sightrag/trackers/bytetrack.py): `sightrag/trackers/bytetrack.py`

### Interfaces

- [Results and timelines](https://github.com/vk-ant/sightrag/blob/main/sightrag/visualizer.py): `sightrag/visualizer.py`
- [CLI](https://github.com/vk-ant/sightrag/blob/main/sightrag/cli.py): `sightrag/cli.py`
- [REST API](https://github.com/vk-ant/sightrag/blob/main/sightrag/api.py): `sightrag/api.py`

### Other components

- **User**

## Connections

- User → SightRAG: uses
- Visual inputs → SightRAG: indexes
- SightRAG → Indexer: dispatches indexing (evidence: `sightrag/core.py`)
- Indexer → Frame extraction: extracts frames (evidence: `sightrag/indexer.py`)
- Indexer → Detection: detects objects
- Indexer → Segmentation: segments inputs
- Indexer → Embedding: embeds regions
- Indexer → OCR: reads text
- Indexer → Object tracking: tracks detections
- SightRAG → Inference backends: selects backend (evidence: `sightrag/core.py`)
- Indexer → Vector and track store: writes vectors
- Indexer → Vector and track store: saves tracks
- SightRAG → Retriever: dispatches queries (evidence: `sightrag/core.py`)
- Retriever → Embedding: embeds query (evidence: `sightrag/retriever.py`)
- Retriever → Detection: detects reference (evidence: `sightrag/retriever.py`)
- Retriever → Vector and track store: searches vectors (evidence: `sightrag/retriever.py`)
- SightRAG → Reranking: reranks results (evidence: `sightrag/core.py`)
- SightRAG → Multimodal understanding: understands queries (evidence: `sightrag/core.py`)
- SightRAG → Vector and track store: reads track history (evidence: `sightrag/core.py`)
- SightRAG → Results and timelines: renders results (evidence: `sightrag/core.py`)
- User → CLI: runs commands
- User → REST API: sends requests
- CLI → SightRAG: invokes features
- REST API → SightRAG: serves operations
