# eshpelin/nurby architecture

> Architecture diagram of the GitHub repository eshpelin/nurby, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-11-19). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/eshpelin/nurby
- Repository: https://github.com/eshpelin/nurby

## Overview

Nurby is a self-hosted camera NVR and AI surveillance platform. Its main workflow takes camera streams through ingestion and perception, turns detections into identities, events, journeys, and searchable descriptions, then exposes those results through the web/API and agent for review or automation. Rules can trigger notifications and physical actions; separate capabilities include Guardian care workflows, voice/audio, MQTT, and external integrations. Some subsystem links are documented but their implementation was not sampled, so those links are dashed and marked unverified.

Main entry points: [`services/perception/pipeline.py`](https://github.com/eshpelin/nurby/blob/main/services/perception/pipeline.py) (Perception pipeline), [`services/ingestion/manager.py`](https://github.com/eshpelin/nurby/blob/main/services/ingestion/manager.py) (Stream ingestion), [`services/api/main.py`](https://github.com/eshpelin/nurby/blob/main/services/api/main.py) (REST and WebSocket API), [`services/agent/llm.py`](https://github.com/eshpelin/nurby/blob/main/services/agent/llm.py) (Ask agent)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_capture["Capture and perception"]
  node_ingestion["Stream ingestion<br/>[manager.py]"]
  node_pipeline["Perception pipeline<br/>[pipeline.py]"]
  node_vlm["Vision descriptions<br/>[vlm.py]"]
  node_audio["Audio processing<br/>[router.py]"]
  node_mainloop["Perception workers<br/>[main.py]"]
end

subgraph group_understanding["Intelligence and history"]
  node_identity["Incidents and journeys"]
  node_journeys["Journey tracking<br/>[journey_tracker.py]"]
  node_search["Semantic search<br/>[query.py]"]
  node_embeddings["Text embeddings<br/>[embeddings.py]"]
  node_agent["Ask agent<br/>[llm.py]"]
end

subgraph group_interaction["User applications"]
  node_web["Web dashboard<br/>[page.tsx]"]
  node_api["REST and WebSocket API<br/>[main.py]"]
  node_ws["Live updates<br/>[ws.py]"]
  node_guardian["Guardian workflows<br/>[alerts.py]"]
  node_voice["Voice conversations<br/>[orchestrator.py]"]
end

subgraph group_automation["Rules and integrations"]
  node_rules["Rule evaluation<br/>[engine.py]"]
  node_actions["Rule actions<br/>[actions.py]"]
  node_mqtt["Home automation MQTT<br/>[publishers.py]"]
  node_mcp["MCP server<br/>[server.py]"]
end

subgraph group_platform["Shared platform"]
  node_database[("Application database<br/>[database.py]")]
  node_models["Domain records"]
  node_settings["Runtime settings<br/>[app_settings.py]"]
  node_budget["Agent budgets<br/>[budget.py]"]
  node_config["Deployment config<br/>[config.py]"]
  node_storage["Media storage paths<br/>[storage_paths.py]"]
end

node_camera(("IP cameras"))

node_camera -->|"sends streams"| node_ingestion
node_ingestion -.->|"feeds frames"| node_pipeline
node_ingestion -->|"starts audio"| node_audio
node_ingestion -->|"publishes updates"| node_ws
node_pipeline -.->|"requests captions"| node_vlm
node_mainloop -->|"runs workers"| node_pipeline
node_mainloop -->|"starts tracker"| node_identity
node_pipeline -->|"evaluates events"| node_rules
node_pipeline -->|"creates vectors"| node_embeddings
node_identity -->|"uses journey data"| node_journeys
node_search -->|"generates vectors"| node_embeddings
node_web -.->|"uses API"| node_api
node_api -->|"serves agent requests"| node_agent
node_api -.->|"serves Guardian routes"| node_guardian
node_api -->|"hosts sockets"| node_ws
node_agent -.->|"answers footage queries"| node_search
node_rules -->|"dispatches actions"| node_actions
node_actions -->|"publishes device actions"| node_mqtt
node_mcp -->|"exposes agent tools"| node_agent
node_pipeline -->|"stores observations"| node_database
node_database -->|"maps records"| node_models
node_settings -->|"reads and writes"| node_database
node_budget -->|"limits usage"| node_agent
node_config -->|"configures connection"| node_database
node_storage -->|"reads override"| node_settings
node_ingestion -.->|"resolves recording roots"| node_storage

click node_ingestion "https://github.com/eshpelin/nurby/blob/main/services/ingestion/manager.py"
click node_pipeline "https://github.com/eshpelin/nurby/blob/main/services/perception/pipeline.py"
click node_vlm "https://github.com/eshpelin/nurby/blob/main/services/perception/vlm.py"
click node_audio "https://github.com/eshpelin/nurby/blob/main/services/perception/audio/router.py"
click node_mainloop "https://github.com/eshpelin/nurby/blob/main/services/perception/main.py"
click node_identity "https://github.com/eshpelin/nurby/blob/main/services/perception/incident_tracker.py"
click node_journeys "https://github.com/eshpelin/nurby/blob/main/services/perception/journey_tracker.py"
click node_search "https://github.com/eshpelin/nurby/blob/main/services/search/query.py"
click node_embeddings "https://github.com/eshpelin/nurby/blob/main/services/search/embeddings.py"
click node_agent "https://github.com/eshpelin/nurby/blob/main/services/agent/llm.py"
click node_web "https://github.com/eshpelin/nurby/blob/main/frontend/src/app/page.tsx"
click node_api "https://github.com/eshpelin/nurby/blob/main/services/api/main.py"
click node_ws "https://github.com/eshpelin/nurby/blob/main/services/api/ws.py"
click node_guardian "https://github.com/eshpelin/nurby/blob/main/services/guardian/alerts.py"
click node_voice "https://github.com/eshpelin/nurby/blob/main/services/voice/orchestrator.py"
click node_rules "https://github.com/eshpelin/nurby/blob/main/services/events/engine.py"
click node_actions "https://github.com/eshpelin/nurby/blob/main/services/events/actions.py"
click node_mqtt "https://github.com/eshpelin/nurby/blob/main/services/integrations/mqtt/publishers.py"
click node_mcp "https://github.com/eshpelin/nurby/blob/main/services/mcp/server.py"
click node_database "https://github.com/eshpelin/nurby/blob/main/shared/database.py"
click node_models "https://github.com/eshpelin/nurby/tree/main/shared/models"
click node_settings "https://github.com/eshpelin/nurby/blob/main/shared/app_settings.py"
click node_budget "https://github.com/eshpelin/nurby/blob/main/services/agent/budget.py"
click node_config "https://github.com/eshpelin/nurby/blob/main/shared/config.py"
click node_storage "https://github.com/eshpelin/nurby/blob/main/shared/storage_paths.py"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_ingestion,node_pipeline,node_vlm,node_audio,node_mainloop toneBlue
class node_identity,node_journeys,node_search,node_embeddings,node_agent toneAmber
class node_web,node_api,node_ws,node_guardian,node_voice toneMint
class node_rules,node_actions,node_mqtt,node_mcp toneRose
class node_database,node_models,node_settings,node_budget,node_config,node_storage,node_camera toneIndigo
```

## Components

### Capture and perception

- [Stream ingestion](https://github.com/eshpelin/nurby/blob/main/services/ingestion/manager.py): `services/ingestion/manager.py`
- [Perception pipeline](https://github.com/eshpelin/nurby/blob/main/services/perception/pipeline.py): `services/perception/pipeline.py`
- [Vision descriptions](https://github.com/eshpelin/nurby/blob/main/services/perception/vlm.py): `services/perception/vlm.py`
- [Audio processing](https://github.com/eshpelin/nurby/blob/main/services/perception/audio/router.py): `services/perception/audio/router.py`
- [Perception workers](https://github.com/eshpelin/nurby/blob/main/services/perception/main.py): `services/perception/main.py`

### Intelligence and history

- [Incidents and journeys](https://github.com/eshpelin/nurby/blob/main/services/perception/incident_tracker.py): `services/perception/incident_tracker.py`
- [Journey tracking](https://github.com/eshpelin/nurby/blob/main/services/perception/journey_tracker.py): `services/perception/journey_tracker.py`
- [Semantic search](https://github.com/eshpelin/nurby/blob/main/services/search/query.py): `services/search/query.py`
- [Text embeddings](https://github.com/eshpelin/nurby/blob/main/services/search/embeddings.py): `services/search/embeddings.py`
- [Ask agent](https://github.com/eshpelin/nurby/blob/main/services/agent/llm.py): `services/agent/llm.py`

### User applications

- [Web dashboard](https://github.com/eshpelin/nurby/blob/main/frontend/src/app/page.tsx): `frontend/src/app/page.tsx`
- [REST and WebSocket API](https://github.com/eshpelin/nurby/blob/main/services/api/main.py): `services/api/main.py`
- [Live updates](https://github.com/eshpelin/nurby/blob/main/services/api/ws.py): `services/api/ws.py`
- [Guardian workflows](https://github.com/eshpelin/nurby/blob/main/services/guardian/alerts.py): `services/guardian/alerts.py`
- [Voice conversations](https://github.com/eshpelin/nurby/blob/main/services/voice/orchestrator.py): `services/voice/orchestrator.py`

### Rules and integrations

- [Rule evaluation](https://github.com/eshpelin/nurby/blob/main/services/events/engine.py): `services/events/engine.py`
- [Rule actions](https://github.com/eshpelin/nurby/blob/main/services/events/actions.py): `services/events/actions.py`
- [Home automation MQTT](https://github.com/eshpelin/nurby/blob/main/services/integrations/mqtt/publishers.py): `services/integrations/mqtt/publishers.py`
- [MCP server](https://github.com/eshpelin/nurby/blob/main/services/mcp/server.py): `services/mcp/server.py`

### Shared platform

- [Application database](https://github.com/eshpelin/nurby/blob/main/shared/database.py): `shared/database.py`
- [Domain records](https://github.com/eshpelin/nurby/tree/main/shared/models): `shared/models`
- [Runtime settings](https://github.com/eshpelin/nurby/blob/main/shared/app_settings.py): `shared/app_settings.py`
- [Agent budgets](https://github.com/eshpelin/nurby/blob/main/services/agent/budget.py): `services/agent/budget.py`
- [Deployment config](https://github.com/eshpelin/nurby/blob/main/shared/config.py): `shared/config.py`
- [Media storage paths](https://github.com/eshpelin/nurby/blob/main/shared/storage_paths.py): `shared/storage_paths.py`

### Other components

- **IP cameras**

## Connections

- IP cameras → Stream ingestion: sends streams (evidence: `README.md`)
- Stream ingestion → Perception pipeline: feeds frames
- Stream ingestion → Audio processing: starts audio (evidence: `services/ingestion/manager.py`)
- Stream ingestion → Live updates: publishes updates (evidence: `services/ingestion/manager.py`)
- Perception pipeline → Vision descriptions: requests captions (evidence: `services/perception/pipeline.py`)
- Perception workers → Perception pipeline: runs workers (evidence: `services/perception/main.py`)
- Perception workers → Incidents and journeys: starts tracker (evidence: `services/perception/main.py`)
- Perception pipeline → Rule evaluation: evaluates events (evidence: `services/perception/pipeline.py`)
- Perception pipeline → Text embeddings: creates vectors (evidence: `services/perception/pipeline.py`)
- Incidents and journeys → Journey tracking: uses journey data (evidence: `services/perception/incident_tracker.py`)
- Semantic search → Text embeddings: generates vectors (evidence: `services/search/query.py`)
- Web dashboard → REST and WebSocket API: uses API
- REST and WebSocket API → Ask agent: serves agent requests (evidence: `services/api/main.py`)
- REST and WebSocket API → Guardian workflows: serves Guardian routes
- REST and WebSocket API → Live updates: hosts sockets (evidence: `services/api/main.py`)
- Ask agent → Semantic search: answers footage queries
- Rule evaluation → Rule actions: dispatches actions (evidence: `services/events/engine.py`)
- Rule actions → Home automation MQTT: publishes device actions (evidence: `services/integrations/mqtt/publishers.py`)
- MCP server → Ask agent: exposes agent tools (evidence: `services/mcp/server.py`)
- Perception pipeline → Application database: stores observations (evidence: `services/perception/pipeline.py`)
- Application database → Domain records: maps records (evidence: `shared/models/__init__.py`)
- Runtime settings → Application database: reads and writes (evidence: `shared/app_settings.py`)
- Agent budgets → Ask agent: limits usage
- Deployment config → Application database: configures connection (evidence: `shared/database.py`)
- Media storage paths → Runtime settings: reads override (evidence: `shared/storage_paths.py`)
- Stream ingestion → Media storage paths: resolves recording roots (evidence: `services/ingestion/manager.py`)
