# mrtzpngrtz/vectorvision architecture

> Architecture diagram of the GitHub repository mrtzpngrtz/vectorvision, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-02-10). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/mrtzpngrtz/vectorvision
- Repository: https://github.com/mrtzpngrtz/vectorvision

## Overview

##### Project type and purpose
- VectorVision is a full-stack desktop application built on Electron.
- Purpose: local, private, GPU-accelerated ML-powered image/video exploration, clustering, tagging, and similarity search.

##### File structure
- Top-level files:
- package.json / package-lock.json (dependencies & scripts)
- electron-main.js (Electron Main Process entry point)
- preload.js (secure IPC bridge)
- public/ (Renderer Process assets):
- index.html, CSS, and multiple JS modules (UI, canvas-renderer, SOM worker)
- server/ (ML processing services):
- index.js (server orchestration)
- analyzer.js (ONNX model inference pipeline)
- som.js (self-organizing map clustering)
- labels.js (tag-assignment logic)
- screenshots/ (documentation)
- Optional: models/ (downloaded ML models at runtime), dist/ (packaged app)

##### README for architectural insights
- Electron Main Process manages windows, IPC, and launches the server code.
- Preload script exposes a limited IPC API to the Renderer.
- Renderer Process (Chromium) handles 2D/3D UI, visualization, user interactions.
- Server modules handle ONNX Runtime inference (SigLIP, DINOv2, Aesthetic Predictor), SOM clustering, and image I/O via Sharp and FFmpeg.

##### Main components
a. Electron Main Process
- electron-main.js
- Coordinates window lifecycle, IPCMain handlers, spawns or requires server modules
b. Preload Bridge
- preload.js
- Exposes sanitized IPC channels (invoke/receive) to the Renderer
c. Renderer Process (UI)
- public/index.html + public/js/*.js
- Canvas renderer, SOM Web Worker, UI controls, sidebars, word-cloud filter
d. ML Server / Processing Layer
- server/index.js – orchestrator
- server/analyzer.js – ONNX Runtime (DirectML/CoreML) calls
- server/som.js – clustering algorithm
- server/labels.js – semantic tagging
e. Local File System & Cache
- User’s image/video libraries on disk
- models/ cache directory for ML model artifacts
f. External Dependencies
- FFmpeg – frame extraction for videos
- ONNX Runtime – GPU-accelerated inference
- Sharp – image decoding/resizing

##### Relationships and data flow
1. User interacts with UI in the Renderer
2. Renderer → (IPC Renderer.invoke) → Preload → (IPC Main.handle) → Electron Main
3. Main → calls server/index.js (in-process require or child process spawn)
4. server modules read images/videos, generate thumbnails, run ONNX models
5. server emits progress and results events → Electron Main → Preload → Renderer
6. Renderer updates 2D/3D visualization, tags, and clusters in real time

##### Key architectural patterns and design principles
- Event-driven, asynchronous IPC for process isolation and security
- Clear separation of concerns: UI (Renderer) vs. system/ML logic (Main + server)
- Worker threads (Web Worker) for CPU-bound clustering in the Renderer
- Local-only architecture: no external APIs, 100% data privacy
- GPU offload for model inference via ONNX Runtime (DirectML/CoreML)

##### Relevant technologies/frameworks
- Electron (Chromium + Node.js integration)
- ONNX Runtime with DirectML/CoreML backends
- SigLIP, DINOv2, Aesthetic Predictor (ONNX models)
- Sharp (Node.js image processing)
- FFmpeg (video frame extraction)
- ml-som (JavaScript SOM implementation)

– Renderer → Preload → Main (invoke)
– Main → Server modules (function call or spawn)
– Server → Main → Renderer (event stream)
– Server → File System (read/write images, models)
- Detail calls out: model download on first run, thumbnail generation, progress events, 2D/3D clustering pipeline.

## Diagram (Mermaid)

```mermaid
flowchart TD
    %% Renderer Layer
    subgraph "Renderer Process (UI)"
        direction TB
        HTML["public/index.html"]:::process
        App["app.js"]:::process
        Canvas["canvas-renderer.js"]:::process
        Worker["som-worker.js"]:::process
        Style["style.css"]:::process
    end

    %% Preload Bridge
    Preload["preload.js"]:::process

    %% Electron Main
    ElectronMain["electron-main.js (Electron Main Process)"]:::process

    %% Server Layer
    subgraph "ML Server / Processing Layer"
        direction TB
        Orchestrator["server/index.js"]:::process
        Analyzer["server/analyzer.js"]:::process
        SOMServer["server/som.js"]:::process
        Labels["server/labels.js"]:::process
    end

    %% Storage Layer
    subgraph "Local Storage"
        direction TB
        FS["User Image/Video Library"]:::storage
        Cache["models/"]:::storage
    end

    %% External Engines
    ONNX["ONNX Runtime"]:::external
    FFmpeg["FFmpeg"]:::external
    Sharp["Sharp"]:::external

    %% Data Flow
    HTML -->|UI Events| Preload
    App -->|Invoke APIs| Preload
    Canvas -->|Draw Frames| App
    Worker -->|Compute Clusters| App
    Style -->|Styling| HTML

    Preload -->|ipcRenderer.invoke| ElectronMain
    ElectronMain -->|require/spawn| Orchestrator

    Orchestrator -->|call inference| Analyzer
    Analyzer -->|GPU offload| ONNX
    Orchestrator -->|cluster data| SOMServer
    Orchestrator -->|assign tags| Labels

    Orchestrator -->|read/write| FS
    Analyzer -->|cache models| Cache

    Orchestrator -->|use| FFmpeg
    Orchestrator -->|use| Sharp

    Analyzer -->|results/events| ElectronMain
    SOMServer -->|results/events| ElectronMain
    Labels -->|results/events| ElectronMain

    ElectronMain -->|ipcMain.handle events| Preload
    Preload -->|emit events| HTML

    %% Click Events
    click HTML "https://github.com/mrtzpngrtz/vectorvision/blob/main/public/index.html"
    click App "https://github.com/mrtzpngrtz/vectorvision/blob/main/public/js/app.js"
    click Canvas "https://github.com/mrtzpngrtz/vectorvision/blob/main/public/js/canvas-renderer.js"
    click Worker "https://github.com/mrtzpngrtz/vectorvision/blob/main/public/js/som-worker.js"
    click Style "https://github.com/mrtzpngrtz/vectorvision/blob/main/public/css/style.css"
    click Preload "https://github.com/mrtzpngrtz/vectorvision/blob/main/preload.js"
    click ElectronMain "https://github.com/mrtzpngrtz/vectorvision/blob/main/electron-main.js"
    click Orchestrator "https://github.com/mrtzpngrtz/vectorvision/blob/main/server/index.js"
    click Analyzer "https://github.com/mrtzpngrtz/vectorvision/blob/main/server/analyzer.js"
    click SOMServer "https://github.com/mrtzpngrtz/vectorvision/blob/main/server/som.js"
    click Labels "https://github.com/mrtzpngrtz/vectorvision/blob/main/server/labels.js"

    %% Styles
    classDef process fill:#ADD8E6,stroke:#333,stroke-width:1px
    classDef storage fill:#90EE90,stroke:#333,stroke-width:1px
    classDef external fill:#FFA500,stroke:#333,stroke-width:1px
```
