# katipally/openlive architecture

> Architecture diagram of the GitHub repository katipally/openlive, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-10-04). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/katipally/openlive
- Repository: https://github.com/katipally/openlive

## Overview

OpenLive is a local-first voice and vision layer for AI agents, delivered through desktop and web interfaces. A person starts a Chat call, speaks, and receives a spoken model or coding-agent response; Flow extends voice actions across apps, while Dictate types speech at the cursor. The README documents the on-device voice loop, camera/screen input, tools, skills, connectors, memory, and history. The graph includes those product branches and sampled internal layers; several service and UI links are documented capabilities, not verified call paths.

Main entry points: [`services/agent/src/server.ts`](https://github.com/katipally/openlive/blob/main/services/agent/src/server.ts) (Local agent service), [`packages/db/src/queries.ts`](https://github.com/katipally/openlive/blob/main/packages/db/src/queries.ts) (App database), [`services/agent/src/capabilities/routes.ts`](https://github.com/katipally/openlive/blob/main/services/agent/src/capabilities/routes.ts) (Tool capabilities), [`apps/web/src/app/api/history/route.ts`](https://github.com/katipally/openlive/blob/main/apps/web/src/app/api/history/route.ts) (History API)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_client["Desktop and UI"]
  node_desktop["Desktop shell<br/>[main.cjs]"]
  node_webui["App interface<br/>[page.tsx]"]
  node_liveui["Live call UI<br/>[InCall.tsx]"]
  node_flowui["Flow mode<br/>[FlowHome.tsx]"]
  node_dictateui["Dictate mode<br/>[DictateHome.tsx]"]
end

subgraph group_conversation["Voice and sessions"]
  node_voicepipeline["Voice pipeline<br/>[pipelineConfig.ts]"]
  node_liveclient["Live session<br/>[useLiveSession.ts]"]
  node_flowruntime["Flow runtime<br/>[flow-runtime.cjs]"]
  node_flowinput["Flow input<br/>[flow-input.cjs]"]
end

subgraph group_agent["Agent capabilities"]
  node_agentservice["Local agent service<br/>[server.ts]"]
  node_agents["Agent adapters<br/>[supervisor.ts]"]
  node_capabilities["Tool capabilities<br/>[routes.ts]"]
  node_connectors["MCP connectors<br/>[manager.ts]"]
  node_skills["Skills<br/>[routes.ts]"]
  node_memory["Memory<br/>[routes.ts]"]
  node_voiceengines["Native voice engines<br/>[engine.ts]"]
end

subgraph group_storage["Local state"]
  node_db[("App database<br/>[queries.ts]")]
  node_flowstore[("Flow session store<br/>[session.ts]")]
  node_history["History API<br/>[route.ts]"]
end

subgraph group_native["Device integration"]
  node_nativeinput["Native input<br/>[coordinator.rs]"]
  node_nativehook["Input hooks<br/>[hook.rs]"]
end

node_person(("Person"))
node_answer["Model or coding agent"]

node_person -->|"opens app"| node_desktop
node_person -->|"uses modes"| node_webui
node_desktop -->|"shows UI"| node_webui
node_person -->|"starts call"| node_liveui
node_liveui -.->|"runs session"| node_liveclient
node_liveclient -.->|"uses voice loop"| node_voicepipeline
node_voicepipeline -->|"sends transcript"| node_answer
node_answer -->|"returns response"| node_liveui
node_person -->|"invokes Flow"| node_flowui
node_flowui -.->|"requests actions"| node_flowruntime
node_flowruntime -->|"uses input"| node_flowinput
node_flowinput -.->|"types at cursor"| node_nativeinput
node_nativeinput -.->|"uses hooks"| node_nativehook
node_person -->|"dictates text"| node_dictateui
node_dictateui -.->|"inserts speech"| node_nativeinput
node_agentservice -.->|"manages agents"| node_agents
node_agents -->|"dispatches turns"| node_answer
node_agentservice -.->|"serves tools"| node_capabilities
node_capabilities -->|"lists connector tools"| node_connectors
node_capabilities -->|"reads settings"| node_db
node_agentservice -.->|"serves skills"| node_skills
node_skills -->|"stores skill settings"| node_db
node_agentservice -.->|"serves memory"| node_memory
node_memory -.->|"uses persistence"| node_db
node_agentservice -.->|"runs native speech"| node_voiceengines
node_voiceengines -->|"reads model settings"| node_db
node_history -->|"reads chats"| node_db
node_history -->|"reads retention"| node_flowstore
node_person -->|"views sessions"| node_history

click node_desktop "https://github.com/katipally/openlive/blob/main/apps/desktop/main.cjs"
click node_webui "https://github.com/katipally/openlive/blob/main/apps/web/src/app/page.tsx"
click node_liveui "https://github.com/katipally/openlive/blob/main/apps/web/src/components/live/InCall.tsx"
click node_flowui "https://github.com/katipally/openlive/blob/main/apps/web/src/components/flow/FlowHome.tsx"
click node_dictateui "https://github.com/katipally/openlive/blob/main/apps/web/src/components/dictate/DictateHome.tsx"
click node_voicepipeline "https://github.com/katipally/openlive/blob/main/apps/web/src/lib/live/pipelineConfig.ts"
click node_liveclient "https://github.com/katipally/openlive/blob/main/apps/web/src/lib/live/useLiveSession.ts"
click node_flowruntime "https://github.com/katipally/openlive/blob/main/apps/desktop/flow-runtime.cjs"
click node_flowinput "https://github.com/katipally/openlive/blob/main/apps/desktop/flow-input.cjs"
click node_agentservice "https://github.com/katipally/openlive/blob/main/services/agent/src/server.ts"
click node_agents "https://github.com/katipally/openlive/blob/main/services/agent/src/agents/supervisor.ts"
click node_capabilities "https://github.com/katipally/openlive/blob/main/services/agent/src/capabilities/routes.ts"
click node_connectors "https://github.com/katipally/openlive/blob/main/services/agent/src/connectors/manager.ts"
click node_skills "https://github.com/katipally/openlive/blob/main/services/agent/src/skills/routes.ts"
click node_memory "https://github.com/katipally/openlive/blob/main/services/agent/src/memory/routes.ts"
click node_voiceengines "https://github.com/katipally/openlive/blob/main/services/agent/src/voice/engine.ts"
click node_db "https://github.com/katipally/openlive/blob/main/packages/db/src/queries.ts"
click node_flowstore "https://github.com/katipally/openlive/blob/main/packages/flow-store/src/session.ts"
click node_history "https://github.com/katipally/openlive/blob/main/apps/web/src/app/api/history/route.ts"
click node_nativeinput "https://github.com/katipally/openlive/blob/main/native/ol-input/src/coordinator.rs"
click node_nativehook "https://github.com/katipally/openlive/blob/main/native/ol-input/src/hook.rs"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_desktop,node_webui,node_liveui,node_flowui,node_dictateui toneBlue
class node_voicepipeline,node_liveclient,node_flowruntime,node_flowinput toneAmber
class node_agentservice,node_agents,node_capabilities,node_connectors,node_skills,node_memory,node_voiceengines toneMint
class node_db,node_flowstore,node_history toneRose
class node_nativeinput,node_nativehook,node_person,node_answer toneIndigo
```

## Components

### Desktop and UI

- [Desktop shell](https://github.com/katipally/openlive/blob/main/apps/desktop/main.cjs): `apps/desktop/main.cjs`
- [App interface](https://github.com/katipally/openlive/blob/main/apps/web/src/app/page.tsx): `apps/web/src/app/page.tsx`
- [Live call UI](https://github.com/katipally/openlive/blob/main/apps/web/src/components/live/InCall.tsx): `apps/web/src/components/live/InCall.tsx`
- [Flow mode](https://github.com/katipally/openlive/blob/main/apps/web/src/components/flow/FlowHome.tsx): `apps/web/src/components/flow/FlowHome.tsx`
- [Dictate mode](https://github.com/katipally/openlive/blob/main/apps/web/src/components/dictate/DictateHome.tsx): `apps/web/src/components/dictate/DictateHome.tsx`

### Voice and sessions

- [Voice pipeline](https://github.com/katipally/openlive/blob/main/apps/web/src/lib/live/pipelineConfig.ts): `apps/web/src/lib/live/pipelineConfig.ts`
- [Live session](https://github.com/katipally/openlive/blob/main/apps/web/src/lib/live/useLiveSession.ts): `apps/web/src/lib/live/useLiveSession.ts`
- [Flow runtime](https://github.com/katipally/openlive/blob/main/apps/desktop/flow-runtime.cjs): `apps/desktop/flow-runtime.cjs`
- [Flow input](https://github.com/katipally/openlive/blob/main/apps/desktop/flow-input.cjs): `apps/desktop/flow-input.cjs`

### Agent capabilities

- [Local agent service](https://github.com/katipally/openlive/blob/main/services/agent/src/server.ts): `services/agent/src/server.ts`
- [Agent adapters](https://github.com/katipally/openlive/blob/main/services/agent/src/agents/supervisor.ts): `services/agent/src/agents/supervisor.ts`
- [Tool capabilities](https://github.com/katipally/openlive/blob/main/services/agent/src/capabilities/routes.ts): `services/agent/src/capabilities/routes.ts`
- [MCP connectors](https://github.com/katipally/openlive/blob/main/services/agent/src/connectors/manager.ts): `services/agent/src/connectors/manager.ts`
- [Skills](https://github.com/katipally/openlive/blob/main/services/agent/src/skills/routes.ts): `services/agent/src/skills/routes.ts`
- [Memory](https://github.com/katipally/openlive/blob/main/services/agent/src/memory/routes.ts): `services/agent/src/memory/routes.ts`
- [Native voice engines](https://github.com/katipally/openlive/blob/main/services/agent/src/voice/engine.ts): `services/agent/src/voice/engine.ts`

### Local state

- [App database](https://github.com/katipally/openlive/blob/main/packages/db/src/queries.ts): `packages/db/src/queries.ts`
- [Flow session store](https://github.com/katipally/openlive/blob/main/packages/flow-store/src/session.ts): `packages/flow-store/src/session.ts`
- [History API](https://github.com/katipally/openlive/blob/main/apps/web/src/app/api/history/route.ts): `apps/web/src/app/api/history/route.ts`

### Device integration

- [Native input](https://github.com/katipally/openlive/blob/main/native/ol-input/src/coordinator.rs): `native/ol-input/src/coordinator.rs`
- [Input hooks](https://github.com/katipally/openlive/blob/main/native/ol-input/src/hook.rs): `native/ol-input/src/hook.rs`

### Other components

- **Person**
- **Model or coding agent**

## Connections

- Person → Desktop shell: opens app (evidence: `README.md`)
- Person → App interface: uses modes (evidence: `README.md`)
- Desktop shell → App interface: shows UI (evidence: `apps/desktop/main.cjs`)
- Person → Live call UI: starts call (evidence: `README.md`)
- Live call UI → Live session: runs session
- Live session → Voice pipeline: uses voice loop
- Voice pipeline → Model or coding agent: sends transcript (evidence: `README.md`)
- Model or coding agent → Live call UI: returns response (evidence: `README.md`)
- Person → Flow mode: invokes Flow (evidence: `README.md`)
- Flow mode → Flow runtime: requests actions
- Flow runtime → Flow input: uses input (evidence: `apps/desktop/flow-runtime.cjs`)
- Flow input → Native input: types at cursor
- Native input → Input hooks: uses hooks
- Person → Dictate mode: dictates text (evidence: `README.md`)
- Dictate mode → Native input: inserts speech
- Local agent service → Agent adapters: manages agents
- Agent adapters → Model or coding agent: dispatches turns (evidence: `README.md`)
- Local agent service → Tool capabilities: serves tools
- Tool capabilities → MCP connectors: lists connector tools (evidence: `services/agent/src/capabilities/routes.ts`)
- Tool capabilities → App database: reads settings (evidence: `services/agent/src/capabilities/routes.ts`)
- Local agent service → Skills: serves skills
- Skills → App database: stores skill settings (evidence: `services/agent/src/skills/routes.ts`)
- Local agent service → Memory: serves memory
- Memory → App database: uses persistence
- Local agent service → Native voice engines: runs native speech
- Native voice engines → App database: reads model settings (evidence: `services/agent/src/voice/engine.ts`)
- History API → App database: reads chats (evidence: `apps/web/src/app/api/history/route.ts`)
- History API → Flow session store: reads retention (evidence: `apps/web/src/app/api/history/route.ts`)
- Person → History API: views sessions (evidence: `README.md`)
