# ssrajadh/sentrysearch architecture

> Architecture diagram of the GitHub repository ssrajadh/sentrysearch, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-04-06). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/ssrajadh/sentrysearch
- Repository: https://github.com/ssrajadh/sentrysearch

## Overview

1. **What this repo is**
   SentrySearch is a Python 3.11+ CLI application for semantic search over video files. Its core architecture is an offline-first indexing pipeline plus a local vector database, with two interchangeable embedding backends: Google Gemini Embedding API or a local Qwen3-VL model.

2. **Primary user-facing boundary: the CLI**
   `sentrysearch/cli.py` is the main entry point. It exposes commands for setup (`init`), indexing (`index`), querying (`search`), index maintenance (`stats`, `remove`, `reset`), and optional Tesla overlay rendering. The CLI orchestrates the rest of the system but does not hold the domain logic itself.

3. **Ingestion pipeline: files → chunks → embeddings**
   The indexing flow starts by recursively discovering supported video files (`.mp4`, `.mov`). `chunker.py` splits each source video into overlapping time windows, with optional preprocessing to downscale and reduce frame rate before model ingestion. This is the first major performance boundary because it determines how much video is sent to the embedder and how many vector records are created.

4. **Backend abstraction for video embeddings**
   The embedding layer is split into a small abstraction and concrete implementations:
   - `base_embedder.py` defines the shared contract.
   - `gemini_embedder.py` calls the Gemini API.
   - `local_embedder.py` runs Qwen3-VL locally, with hardware-aware model selection and optional quantization.
   - `embedder.py` appears to be the backend selection/facade layer.
   
   This boundary matters because the rest of the system treats embeddings as backend-agnostic, while the index stores enough metadata to remember which backend/model created it.

5. **Local persistence and index isolation**
   `store.py` manages the on-disk vector store, backed by ChromaDB per the README. It stores chunk embeddings plus metadata such as source path, chunk time range, and backend/model identity. Search auto-detects the correct backend/model from the stored index, which implies the store is also the authority for index compatibility and separation between models.

6. **Search flow: text query → vector match → ranked results**
   `search.py` embeds the user’s text query into the same vector space as the indexed video chunks, then retrieves the nearest neighbors from the vector store. The output is a ranked list with similarity scores and time ranges. This is the second major system boundary: text input is transformed into retrieval over previously embedded video segments.

7. **Clip generation: result → trimmed output file**
   `trimmer.py` takes the winning match, maps it back to the original source video, and extracts the corresponding time window into a standalone clip. The README makes clear this is usually automatic, but can be skipped with a flag. This stage depends on video tooling (`ffmpeg` or bundled `imageio-ffmpeg`) and is separate from search so retrieval and media extraction can evolve independently.

8. **Optional Tesla-specific metadata overlay**
   Tesla dashcam footage can include embedded telemetry. `dashcam.proto` and `dashcam_pb2.py` define/consume that structured metadata, while `metadata.py` extracts it and `overlay.py` burns it into the trimmed clip as an on-video HUD. This is an optional post-processing branch after trimming, and it introduces an external dependency only for reverse geocoding when city/road names are desired.

9. **Operational behavior and environment dependencies**
   The system is designed to run locally. It uses `.env` for API credentials, `uv` for installation/tooling, ChromaDB for persistence, and `ffmpeg` for decoding/trimming. The local backend additionally depends on GPU-capable ML runtimes (CUDA or Apple Metal) and, on macOS, a system FFmpeg installation for the video processor.

10. **Main data flows to capture in an architecture diagram**
   There are two end-to-end flows worth diagramming:
   - **Indexing:** video files on disk → chunking/preprocessing → embedding backend → local vector store.
   - **Searching:** user text query → text embedding → vector retrieval from the same store → optional trim from original video → optional Tesla overlay → saved clip.
   
   The key architectural boundary is that the search engine never transcribes video; it compares embeddings directly, and the persistence layer keeps backend/model-specific indexes isolated so incompatible embeddings are not mixed.

Main entry points: [`sentrysearch/embedder.py`](https://github.com/ssrajadh/sentrysearch/blob/master/sentrysearch/embedder.py) (Backend select), [`sentrysearch/search.py`](https://github.com/ssrajadh/sentrysearch/blob/master/sentrysearch/search.py) (Search), [`sentrysearch/trimmer.py`](https://github.com/ssrajadh/sentrysearch/blob/master/sentrysearch/trimmer.py) (Trimmer), [`sentrysearch/store.py`](https://github.com/ssrajadh/sentrysearch/blob/master/sentrysearch/store.py) (Vector store)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_group_cli["CLI surface"]
  node_node_cli["CLI<br/>command entrypoint<br/>[cli.py]"]
  node_node_text_query["Text query<br/>user input"]
end

subgraph group_group_ingest["Indexing pipeline"]
  node_node_chunker["Chunker<br/>video segmentation<br/>[chunker.py]"]
  node_node_preproc["Preprocess<br/>video prep<br/>[chunker.py]"]
  node_node_video_files[("Video files<br/>source media")]
end

subgraph group_group_embed["Embedding backends"]
  node_node_embed_iface["Embed API<br/>embedder contract<br/>[base_embedder.py]"]
  node_node_embed_select["Backend select<br/>embedder facade<br/>[embedder.py]"]
  node_node_gemini["Gemini<br/>cloud embedder<br/>[gemini_embedder.py]"]
  node_node_local["Local Qwen<br/>local embedder<br/>[local_embedder.py]"]
end

subgraph group_group_store["Local index"]
  node_node_store[("Vector store<br/>chroma index<br/>[store.py]")]
  node_node_index_meta["Index metadata<br/>compatibility state<br/>[store.py]"]
end

subgraph group_group_retrieve["Retrieval and output"]
  node_node_search["Search<br/>retrieval engine<br/>[search.py]"]
  node_node_results["Ranked hits<br/>retrieval results<br/>[search.py]"]
  node_node_trimmer["Trimmer<br/>clip extractor<br/>[trimmer.py]"]
  node_node_overlay["Tesla overlay<br/>hud renderer<br/>[overlay.py]"]
  node_node_metadata["Metadata<br/>telemetry parser<br/>[metadata.py]"]
  node_node_proto["Dashcam proto<br/>schema<br/>[dashcam.proto]"]
  node_node_pb2["Proto bindings<br/>generated code<br/>[dashcam_pb2.py]"]
  node_node_clip[("Saved clip<br/>output media")]
end

node_node_cli -->|"index"| node_node_chunker
node_node_cli -->|"search"| node_node_search
node_node_cli -->|"clip"| node_node_trimmer
node_node_video_files -->|"scan"| node_node_chunker
node_node_chunker -->|"prepare"| node_node_preproc
node_node_preproc -->|"chunks"| node_node_embed_select
node_node_embed_iface -->|"contract"| node_node_embed_select
node_node_embed_select -->|"gemini"| node_node_gemini
node_node_embed_select -->|"local"| node_node_local
node_node_gemini -->|"store"| node_node_store
node_node_local -->|"store"| node_node_store
node_node_store -->|"compat"| node_node_index_meta
node_node_text_query -->|"embed query"| node_node_search
node_node_search -->|"retrieve"| node_node_store
node_node_search -->|"rank"| node_node_results
node_node_results -->|"best hit"| node_node_trimmer
node_node_trimmer -->|"save"| node_node_clip
node_node_metadata -->|"parse"| node_node_proto
node_node_proto -->|"bind"| node_node_pb2
node_node_trimmer -->|"optional"| node_node_overlay
node_node_metadata -->|"telemetry"| node_node_overlay
node_node_overlay -->|"burn in"| node_node_clip

click node_node_cli "https://github.com/ssrajadh/sentrysearch/blob/master/sentrysearch/cli.py"
click node_node_chunker "https://github.com/ssrajadh/sentrysearch/blob/master/sentrysearch/chunker.py"
click node_node_preproc "https://github.com/ssrajadh/sentrysearch/blob/master/sentrysearch/chunker.py"
click node_node_embed_iface "https://github.com/ssrajadh/sentrysearch/blob/master/sentrysearch/base_embedder.py"
click node_node_embed_select "https://github.com/ssrajadh/sentrysearch/blob/master/sentrysearch/embedder.py"
click node_node_gemini "https://github.com/ssrajadh/sentrysearch/blob/master/sentrysearch/gemini_embedder.py"
click node_node_local "https://github.com/ssrajadh/sentrysearch/blob/master/sentrysearch/local_embedder.py"
click node_node_store "https://github.com/ssrajadh/sentrysearch/blob/master/sentrysearch/store.py"
click node_node_index_meta "https://github.com/ssrajadh/sentrysearch/blob/master/sentrysearch/store.py"
click node_node_search "https://github.com/ssrajadh/sentrysearch/blob/master/sentrysearch/search.py"
click node_node_results "https://github.com/ssrajadh/sentrysearch/blob/master/sentrysearch/search.py"
click node_node_trimmer "https://github.com/ssrajadh/sentrysearch/blob/master/sentrysearch/trimmer.py"
click node_node_overlay "https://github.com/ssrajadh/sentrysearch/blob/master/sentrysearch/overlay.py"
click node_node_metadata "https://github.com/ssrajadh/sentrysearch/blob/master/sentrysearch/metadata.py"
click node_node_proto "https://github.com/ssrajadh/sentrysearch/blob/master/sentrysearch/dashcam.proto"
click node_node_pb2 "https://github.com/ssrajadh/sentrysearch/blob/master/sentrysearch/dashcam_pb2.py"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_node_cli,node_node_text_query toneBlue
class node_node_chunker,node_node_preproc,node_node_video_files toneAmber
class node_node_embed_iface,node_node_embed_select,node_node_gemini,node_node_local toneMint
class node_node_store,node_node_index_meta toneRose
class node_node_search,node_node_results,node_node_trimmer,node_node_overlay,node_node_metadata,node_node_proto,node_node_pb2,node_node_clip toneIndigo
```

## Components

### CLI surface

User commands and orchestration entrypoint

- [CLI](https://github.com/ssrajadh/sentrysearch/blob/master/sentrysearch/cli.py): command entrypoint · Main command dispatcher for init, index, search, and maintenance · `sentrysearch/cli.py`
- **Text query**: user input · Natural-language search prompt entered through the CLI

### Indexing pipeline

Video discovery and chunk preparation before embedding

- [Chunker](https://github.com/ssrajadh/sentrysearch/blob/master/sentrysearch/chunker.py): video segmentation · Finds supported videos and splits them into overlapping time windows · `sentrysearch/chunker.py`
- [Preprocess](https://github.com/ssrajadh/sentrysearch/blob/master/sentrysearch/chunker.py): video prep · Optional downscaling and frame-rate reduction before embedding · `sentrysearch/chunker.py`
- **Video files**: source media · Supported input videos discovered on disk

### Embedding backends

Backend-agnostic embedding contract plus concrete implementations

- [Embed API](https://github.com/ssrajadh/sentrysearch/blob/master/sentrysearch/base_embedder.py): embedder contract · Shared interface for turning video chunks or text into vectors · `sentrysearch/base_embedder.py`
- [Backend select](https://github.com/ssrajadh/sentrysearch/blob/master/sentrysearch/embedder.py): embedder facade · Chooses the active embedding backend and exposes a unified entrypoint · `sentrysearch/embedder.py`
- [Gemini](https://github.com/ssrajadh/sentrysearch/blob/master/sentrysearch/gemini_embedder.py): cloud embedder · Calls the Google Gemini embedding API · `sentrysearch/gemini_embedder.py`
- [Local Qwen](https://github.com/ssrajadh/sentrysearch/blob/master/sentrysearch/local_embedder.py): local embedder · Runs Qwen3-VL locally with hardware-aware model selection and optional quantization · `sentrysearch/local_embedder.py`

### Local index

On-disk vector store and index compatibility boundaries

- [Vector store](https://github.com/ssrajadh/sentrysearch/blob/master/sentrysearch/store.py): chroma index · Persists chunk embeddings and metadata in a local ChromaDB-backed index · `sentrysearch/store.py`
- [Index metadata](https://github.com/ssrajadh/sentrysearch/blob/master/sentrysearch/store.py): compatibility state · Stores backend, model, source path, and chunk range needed to keep indexes isolated · `sentrysearch/store.py`

### Retrieval and output

Search, trimming, and optional Tesla-specific overlay

- [Search](https://github.com/ssrajadh/sentrysearch/blob/master/sentrysearch/search.py): retrieval engine · Embeds a text query and retrieves nearest video chunks with scores · `sentrysearch/search.py`
- [Ranked hits](https://github.com/ssrajadh/sentrysearch/blob/master/sentrysearch/search.py): retrieval results · Similarity-ranked matches with source paths and time ranges · `sentrysearch/search.py`
- [Trimmer](https://github.com/ssrajadh/sentrysearch/blob/master/sentrysearch/trimmer.py): clip extractor · Cuts the winning time window from the original source video · `sentrysearch/trimmer.py`
- [Tesla overlay](https://github.com/ssrajadh/sentrysearch/blob/master/sentrysearch/overlay.py): hud renderer · Optionally burns Tesla telemetry metadata into the trimmed clip · `sentrysearch/overlay.py`
- [Metadata](https://github.com/ssrajadh/sentrysearch/blob/master/sentrysearch/metadata.py): telemetry parser · Extracts structured Tesla dashcam metadata for overlay rendering · `sentrysearch/metadata.py`
- [Dashcam proto](https://github.com/ssrajadh/sentrysearch/blob/master/sentrysearch/dashcam.proto): schema · Protobuf schema and generated bindings for Tesla dashcam metadata · `sentrysearch/dashcam.proto`
- [Proto bindings](https://github.com/ssrajadh/sentrysearch/blob/master/sentrysearch/dashcam_pb2.py): generated code · Generated protobuf module used by the metadata parser · `sentrysearch/dashcam_pb2.py`
- **Saved clip**: output media · Final trimmed clip, optionally with an overlay burned in

## Connections

- CLI → Chunker: index · Starts indexing over discovered video files
- CLI → Search: search · Submits the user query to retrieval
- CLI → Trimmer: clip · Triggers extraction of a chosen result
- Video files → Chunker: scan · Recursively discovers supported video inputs
- Chunker → Preprocess: prepare · Optionally normalizes videos before embedding
- Preprocess → Backend select: chunks · Sends prepared chunks to the active backend
- Embed API → Backend select: contract · Defines the common embedding interface
- Backend select → Gemini: gemini · Routes embedding work to the cloud backend
- Backend select → Local Qwen: local · Routes embedding work to the local model backend
- Gemini → Vector store: store · Writes vectors and metadata to the index
- Local Qwen → Vector store: store · Writes vectors and metadata to the index
- Vector store → Index metadata: compat · Keeps backend and model identity with each index
- Text query → Search: embed query · Transforms text into the same vector space as indexed chunks
- Search → Vector store: retrieve · Fetches nearest neighbors from the local vector store
- Search → Ranked hits: rank · Produces scored matches and source time ranges
- Ranked hits → Trimmer: best hit · Uses the winning match to locate the source segment
- Trimmer → Saved clip: save · Extracts the segment into a standalone clip
- Metadata → Dashcam proto: parse · Interprets dashcam telemetry using the protobuf schema
- Dashcam proto → Proto bindings: bind · Generated protobuf bindings back the metadata reader
- Trimmer → Tesla overlay: optional · Hands trimmed video to the overlay stage when enabled
- Metadata → Tesla overlay: telemetry · Provides location and vehicle metadata for HUD rendering
- Tesla overlay → Saved clip: burn in · Produces the final annotated clip
