# xiaotianfotos/indexed architecture

> Architecture diagram of the GitHub repository xiaotianfotos/indexed, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-09-30). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/xiaotianfotos/indexed
- Repository: https://github.com/xiaotianfotos/indexed

## Overview

Indexed is a local-first multimodal library for explicitly added files and user-captured web video. People and agents configure an embedding backend, scan assets into model-isolated vector indexes, then search and return source paths, timestamps, and text. The map covers the documented Dashboard, CLI, browser capture, local storage, and Apple-native embedding paths. Several route and storage implementations are unsampled; edges are limited to documented relationships or visible imports and calls.

Main entry points: [`apps/server/src/index.ts`](https://github.com/xiaotianfotos/indexed/blob/main/apps/server/src/index.ts) (Local HTTP service), [`packages/core/src/ingest.ts`](https://github.com/xiaotianfotos/indexed/blob/main/packages/core/src/ingest.ts) (Asset ingestion), [`packages/core/src/asset-library.ts`](https://github.com/xiaotianfotos/indexed/blob/main/packages/core/src/asset-library.ts) (Asset library), [`apps/server/src/ingest-test.ts`](https://github.com/xiaotianfotos/indexed/blob/main/apps/server/src/ingest-test.ts) (Ingest test runner)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_access["User access"]
  node_dashboard["Dashboard<br/>[main.ts]"]
  node_cli["CLI<br/>[index.ts]"]
  node_extension["Browser extension<br/>[service-worker.ts]"]
  node_capture["Video capture<br/>[content-script.ts]"]
end

subgraph group_service["Application service"]
  node_server["Local HTTP service<br/>[index.ts]"]
  node_config["Profiles and config<br/>[index.ts]"]
  node_contracts["Shared contracts<br/>[index.ts]"]
  node_ingesttest["Ingest test runner<br/>[ingest-test.ts]"]
end

subgraph group_indexing["Ingest and search"]
  node_ingest["Asset ingestion<br/>[ingest.ts]"]
  node_assetlibrary["Asset library<br/>[asset-library.ts]"]
  node_chunks["Asset chunks<br/>[asset-chunks.ts]"]
  node_documents["Document assets<br/>[document-assets.ts]"]
  node_performance["Ingest performance"]
  node_localsearch["Local file search<br/>[search-assets.ts]"]
  node_library["Library operations<br/>[library.ts]"]
  node_localvectors[("Local vector store<br/>[local-vectors.ts]")]
end

subgraph group_embedding["Embedding backends"]
  node_clients["Embedding transport"]
  node_applebackend["Apple backend adapter<br/>[index.ts]"]
  node_applehelper["Native service<br/>[ServiceMain.swift]"]
  node_router["Inference router"]
  node_mlx["MLX embedding engine"]
  node_coreml["Core ML decoder"]
  node_vision["Vision patches"]
end

node_person(("Local user"))
node_agent(("AI agent"))

node_person -->|"uses"| node_dashboard
node_person -->|"uses"| node_cli
node_agent -->|"invokes"| node_cli
node_person -->|"selects video"| node_capture
node_capture -.->|"sends captures"| node_extension
node_dashboard -.->|"calls API"| node_server
node_cli -->|"uses service"| node_server
node_extension -.->|"sends requests"| node_server
node_server -->|"loads profiles"| node_config
node_server -.->|"uses protocol"| node_contracts
node_server -->|"dispatches ingest"| node_ingest
node_server -->|"dispatches search"| node_localsearch
node_server -->|"forwards embeddings"| node_clients
node_server -->|"manages runtime"| node_applebackend
node_server -->|"runs tests"| node_ingesttest
node_ingest -->|"indexes assets"| node_assetlibrary
node_assetlibrary -->|"uses chunks"| node_chunks
node_assetlibrary -->|"uses extraction"| node_documents
node_assetlibrary -->|"records throughput"| node_performance
node_ingest -.->|"stores vectors"| node_localvectors
node_localsearch -->|"reads assets"| node_assetlibrary
node_localsearch -->|"searches library"| node_library
node_ingest -->|"requests embeddings"| node_clients
node_applebackend -.->|"starts helper"| node_applehelper
node_applehelper -->|"creates router"| node_router
node_router -.->|"routes inference"| node_mlx
node_mlx -->|"uses decoder"| node_coreml
node_mlx -->|"prepares patches"| node_vision
node_ingesttest -->|"checks profile"| node_config
node_ingesttest -->|"scans library"| node_ingest

click node_dashboard "https://github.com/xiaotianfotos/indexed/blob/main/apps/dashboard/src/main.ts"
click node_cli "https://github.com/xiaotianfotos/indexed/blob/main/apps/cli/src/index.ts"
click node_extension "https://github.com/xiaotianfotos/indexed/blob/main/apps/extension/src/service-worker.ts"
click node_capture "https://github.com/xiaotianfotos/indexed/blob/main/apps/extension/src/content-script.ts"
click node_server "https://github.com/xiaotianfotos/indexed/blob/main/apps/server/src/index.ts"
click node_config "https://github.com/xiaotianfotos/indexed/blob/main/packages/config/src/index.ts"
click node_contracts "https://github.com/xiaotianfotos/indexed/blob/main/packages/contracts/src/index.ts"
click node_ingest "https://github.com/xiaotianfotos/indexed/blob/main/packages/core/src/ingest.ts"
click node_assetlibrary "https://github.com/xiaotianfotos/indexed/blob/main/packages/core/src/asset-library.ts"
click node_chunks "https://github.com/xiaotianfotos/indexed/blob/main/packages/core/src/asset-chunks.ts"
click node_documents "https://github.com/xiaotianfotos/indexed/blob/main/packages/core/src/document-assets.ts"
click node_performance "https://github.com/xiaotianfotos/indexed/blob/main/packages/core/src/ingest-performance.ts"
click node_localsearch "https://github.com/xiaotianfotos/indexed/blob/main/packages/core/src/search-assets.ts"
click node_library "https://github.com/xiaotianfotos/indexed/blob/main/packages/core/src/library.ts"
click node_localvectors "https://github.com/xiaotianfotos/indexed/blob/main/packages/clients/src/local-vectors.ts"
click node_clients "https://github.com/xiaotianfotos/indexed/blob/main/packages/clients/src/embedding-transport.ts"
click node_applebackend "https://github.com/xiaotianfotos/indexed/blob/main/packages/apple-embedding-backend/src/index.ts"
click node_applehelper "https://github.com/xiaotianfotos/indexed/blob/main/native/apple-embedding/swift/Sources/IndexedAppleEmbeddingService/ServiceMain.swift"
click node_router "https://github.com/xiaotianfotos/indexed/blob/main/native/apple-embedding/swift/Sources/AppleEmbeddingCore/ServiceRouter.swift"
click node_mlx "https://github.com/xiaotianfotos/indexed/blob/main/native/apple-embedding/swift/Sources/AppleEmbeddingCore/MLXWeMMEngine.swift"
click node_coreml "https://github.com/xiaotianfotos/indexed/blob/main/native/apple-embedding/swift/Sources/AppleEmbeddingCore/CoreMLDecoderBundle.swift"
click node_vision "https://github.com/xiaotianfotos/indexed/blob/main/native/apple-embedding/swift/Sources/AppleEmbeddingCore/VisionPatches.swift"
click node_ingesttest "https://github.com/xiaotianfotos/indexed/blob/main/apps/server/src/ingest-test.ts"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_dashboard,node_cli,node_extension,node_capture,node_person toneBlue
class node_server,node_config,node_contracts,node_ingesttest toneAmber
class node_ingest,node_assetlibrary,node_chunks,node_documents,node_performance,node_localsearch,node_library,node_localvectors toneMint
class node_clients,node_applebackend,node_applehelper,node_router,node_mlx,node_coreml,node_vision toneRose
class node_agent toneIndigo
```

## Components

### User access

- [Dashboard](https://github.com/xiaotianfotos/indexed/blob/main/apps/dashboard/src/main.ts): `apps/dashboard/src/main.ts`
- [CLI](https://github.com/xiaotianfotos/indexed/blob/main/apps/cli/src/index.ts): `apps/cli/src/index.ts`
- [Browser extension](https://github.com/xiaotianfotos/indexed/blob/main/apps/extension/src/service-worker.ts): `apps/extension/src/service-worker.ts`
- [Video capture](https://github.com/xiaotianfotos/indexed/blob/main/apps/extension/src/content-script.ts): `apps/extension/src/content-script.ts`

### Application service

- [Local HTTP service](https://github.com/xiaotianfotos/indexed/blob/main/apps/server/src/index.ts): `apps/server/src/index.ts`
- [Profiles and config](https://github.com/xiaotianfotos/indexed/blob/main/packages/config/src/index.ts): `packages/config/src/index.ts`
- [Shared contracts](https://github.com/xiaotianfotos/indexed/blob/main/packages/contracts/src/index.ts): `packages/contracts/src/index.ts`
- [Ingest test runner](https://github.com/xiaotianfotos/indexed/blob/main/apps/server/src/ingest-test.ts): `apps/server/src/ingest-test.ts`

### Ingest and search

- [Asset ingestion](https://github.com/xiaotianfotos/indexed/blob/main/packages/core/src/ingest.ts): `packages/core/src/ingest.ts`
- [Asset library](https://github.com/xiaotianfotos/indexed/blob/main/packages/core/src/asset-library.ts): `packages/core/src/asset-library.ts`
- [Asset chunks](https://github.com/xiaotianfotos/indexed/blob/main/packages/core/src/asset-chunks.ts): `packages/core/src/asset-chunks.ts`
- [Document assets](https://github.com/xiaotianfotos/indexed/blob/main/packages/core/src/document-assets.ts): `packages/core/src/document-assets.ts`
- [Ingest performance](https://github.com/xiaotianfotos/indexed/blob/main/packages/core/src/ingest-performance.ts): `packages/core/src/ingest-performance.ts`
- [Local file search](https://github.com/xiaotianfotos/indexed/blob/main/packages/core/src/search-assets.ts): `packages/core/src/search-assets.ts`
- [Library operations](https://github.com/xiaotianfotos/indexed/blob/main/packages/core/src/library.ts): `packages/core/src/library.ts`
- [Local vector store](https://github.com/xiaotianfotos/indexed/blob/main/packages/clients/src/local-vectors.ts): `packages/clients/src/local-vectors.ts`

### Embedding backends

- [Embedding transport](https://github.com/xiaotianfotos/indexed/blob/main/packages/clients/src/embedding-transport.ts): `packages/clients/src/embedding-transport.ts`
- [Apple backend adapter](https://github.com/xiaotianfotos/indexed/blob/main/packages/apple-embedding-backend/src/index.ts): `packages/apple-embedding-backend/src/index.ts`
- [Native service](https://github.com/xiaotianfotos/indexed/blob/main/native/apple-embedding/swift/Sources/IndexedAppleEmbeddingService/ServiceMain.swift): `native/apple-embedding/swift/Sources/IndexedAppleEmbeddingService/ServiceMain.swift`
- [Inference router](https://github.com/xiaotianfotos/indexed/blob/main/native/apple-embedding/swift/Sources/AppleEmbeddingCore/ServiceRouter.swift): `native/apple-embedding/swift/Sources/AppleEmbeddingCore/ServiceRouter.swift`
- [MLX embedding engine](https://github.com/xiaotianfotos/indexed/blob/main/native/apple-embedding/swift/Sources/AppleEmbeddingCore/MLXWeMMEngine.swift): `native/apple-embedding/swift/Sources/AppleEmbeddingCore/MLXWeMMEngine.swift`
- [Core ML decoder](https://github.com/xiaotianfotos/indexed/blob/main/native/apple-embedding/swift/Sources/AppleEmbeddingCore/CoreMLDecoderBundle.swift): `native/apple-embedding/swift/Sources/AppleEmbeddingCore/CoreMLDecoderBundle.swift`
- [Vision patches](https://github.com/xiaotianfotos/indexed/blob/main/native/apple-embedding/swift/Sources/AppleEmbeddingCore/VisionPatches.swift): `native/apple-embedding/swift/Sources/AppleEmbeddingCore/VisionPatches.swift`

### Other components

- **Local user**
- **AI agent**

## Connections

- Local user → Dashboard: uses (evidence: `README.md`)
- Local user → CLI: uses (evidence: `README.md`)
- AI agent → CLI: invokes (evidence: `README.md`)
- Local user → Video capture: selects video (evidence: `README.md`)
- Video capture → Browser extension: sends captures
- Dashboard → Local HTTP service: calls API
- CLI → Local HTTP service: uses service (evidence: `apps/cli/src/index.ts`)
- Browser extension → Local HTTP service: sends requests
- Local HTTP service → Profiles and config: loads profiles (evidence: `apps/server/src/index.ts`)
- Local HTTP service → Shared contracts: uses protocol
- Local HTTP service → Asset ingestion: dispatches ingest (evidence: `apps/server/src/index.ts`)
- Local HTTP service → Local file search: dispatches search (evidence: `apps/server/src/index.ts`)
- Local HTTP service → Embedding transport: forwards embeddings (evidence: `apps/server/src/index.ts`)
- Local HTTP service → Apple backend adapter: manages runtime (evidence: `apps/server/src/index.ts`)
- Local HTTP service → Ingest test runner: runs tests (evidence: `apps/server/src/index.ts`)
- Asset ingestion → Asset library: indexes assets (evidence: `packages/core/src/asset-library.ts`)
- Asset library → Asset chunks: uses chunks (evidence: `packages/core/src/asset-library.ts`)
- Asset library → Document assets: uses extraction (evidence: `packages/core/src/asset-library.ts`)
- Asset library → Ingest performance: records throughput (evidence: `packages/core/src/asset-library.ts`)
- Asset ingestion → Local vector store: stores vectors
- Local file search → Asset library: reads assets (evidence: `packages/core/src/search-assets.ts`)
- Local file search → Library operations: searches library (evidence: `packages/core/src/search-assets.ts`)
- Asset ingestion → Embedding transport: requests embeddings (evidence: `packages/core/src/ingest.ts`)
- Apple backend adapter → Native service: starts helper
- Native service → Inference router: creates router (evidence: `native/apple-embedding/swift/Sources/IndexedAppleEmbeddingService/ServiceMain.swift`)
- Inference router → MLX embedding engine: routes inference
- MLX embedding engine → Core ML decoder: uses decoder (evidence: `native/apple-embedding/swift/Sources/AppleEmbeddingCore/MLXWeMMEngine.swift`)
- MLX embedding engine → Vision patches: prepares patches (evidence: `native/apple-embedding/swift/Sources/AppleEmbeddingCore/MLXWeMMEngine.swift`)
- Ingest test runner → Profiles and config: checks profile (evidence: `apps/server/src/ingest-test.ts`)
- Ingest test runner → Asset ingestion: scans library (evidence: `apps/server/src/ingest-test.ts`)