# s-fetah/jarvis architecture

> Architecture diagram of the GitHub repository s-fetah/jarvis, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-11-22). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/s-fetah/jarvis
- Repository: https://github.com/s-fetah/jarvis

## Overview

Jarvis is an experimental local-first Windows desktop assistant. The main workflow is user chat through the desktop UI to the agent orchestrator, which plans tasks, requests model responses, invokes tools, and can seek approval; results and progress return to the UI. The repository also includes mascot and quick-chat surfaces, workspace previews, persistent memory, and voice input/output. The README documents several capabilities whose runtime wiring is not shown in the excerpts, so those links are omitted or marked unverified rather than inferred.

Main entry points: [`packages/core/src/agent/agent-orchestrator.ts`](https://github.com/s-fetah/jarvis/blob/main/packages/core/src/agent/agent-orchestrator.ts) (Agent orchestrator), [`apps/desktop/src/renderer/src/App.tsx`](https://github.com/s-fetah/jarvis/blob/main/apps/desktop/src/renderer/src/App.tsx) (Desktop chat shell), [`packages/core/src/tools/tool-executor.ts`](https://github.com/s-fetah/jarvis/blob/main/packages/core/src/tools/tool-executor.ts) (Tool executor), [`packages/voice/src/voice-manager.ts`](https://github.com/s-fetah/jarvis/blob/main/packages/voice/src/voice-manager.ts) (Voice manager)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_desktop["Desktop surfaces"]
  node_app["Desktop chat shell<br/>[App.tsx]"]
  node_quickchat["Quick chat<br/>[QuickChatApp.tsx]"]
  node_mascot["Mascot window<br/>[mascot-manager.ts]"]
  node_settings["Settings surface<br/>[SettingsApp.tsx]"]
  node_workspace["Workspace canvas"]
  node_promptcontrols["Prompt controls"]
  node_plan["Plan card<br/>[PlanCard.tsx]"]
  node_audio_ui["Audio capture<br/>[audio-service.ts]"]
end

subgraph group_agent["Agent runtime"]
  node_orchestrator["Agent orchestrator"]
  node_prompt["Prompt engine<br/>[prompt-engine.ts]"]
  node_model["Model adapter<br/>[model-adapter.ts]"]
  node_providers["Model providers"]
  node_protocol["Shared protocol<br/>[types.ts]"]
end

subgraph group_tools["Tools and policy"]
  node_executor["Tool executor<br/>[tool-executor.ts]"]
  node_registry["Tool registry<br/>[tool-registry.ts]"]
  node_policy["Approval policy<br/>[policy-engine.ts]"]
  node_process["Process manager<br/>[process-manager.ts]"]
  node_browser["Browser automation<br/>[browser.ts]"]
end

subgraph group_voice["Voice services"]
  node_voice_manager["Voice manager<br/>[voice-manager.ts]"]
  node_vad["Speech detection"]
  node_silero["Silero VAD"]
  node_tts["Speech synthesis<br/>[tts-dispatcher.ts]"]
  node_audio_cache[("Audio cache")]
end

subgraph group_state["Persistence and learning"]
  node_memory[("User memory<br/>[user-memory.ts]")]
  node_learning["Learning engine<br/>[learning-engine.ts]"]
  node_database[("Conversation storage<br/>[database.ts]")]
end

node_user(("User"))

node_user -->|"submits tasks"| node_app
node_app -.->|"requests work"| node_orchestrator
node_orchestrator -->|"requests models"| node_model
node_orchestrator -->|"invokes tools"| node_executor
node_orchestrator -->|"consults tools"| node_registry
node_orchestrator -.->|"builds prompts"| node_prompt
node_prompt -->|"loads profile"| node_memory
node_prompt -->|"checks safety"| node_protocol
node_model -.->|"selects provider"| node_providers
node_executor -->|"checks permissions"| node_policy
node_executor -->|"manages processes"| node_process
node_executor -->|"runs browser tasks"| node_browser
node_executor -->|"uses definitions"| node_registry
node_orchestrator -->|"learns conventions"| node_learning
node_learning -->|"updates notes"| node_memory
node_app -->|"renders canvas"| node_workspace
node_app -->|"renders plans"| node_plan
node_quickchat -->|"renders controls"| node_promptcontrols
node_promptcontrols -->|"uses audio"| node_audio_ui
node_settings -.->|"configures voice"| node_audio_ui
node_audio_ui -.->|"sends audio"| node_voice_manager
node_voice_manager -->|"detects boundaries"| node_vad
node_vad -->|"uses detector"| node_silero
node_voice_manager -->|"dispatches synthesis"| node_tts
node_voice_manager -->|"caches speech"| node_audio_cache
node_orchestrator -.->|"persists history"| node_database
node_mascot -->|"opens drawer"| node_quickchat
node_app -->|"uses shared types"| node_protocol

click node_app "https://github.com/s-fetah/jarvis/blob/main/apps/desktop/src/renderer/src/App.tsx"
click node_quickchat "https://github.com/s-fetah/jarvis/blob/main/apps/desktop/src/renderer/src/quickchat/QuickChatApp.tsx"
click node_mascot "https://github.com/s-fetah/jarvis/blob/main/apps/desktop/src/main/windows/mascot-manager.ts"
click node_settings "https://github.com/s-fetah/jarvis/blob/main/apps/desktop/src/renderer/src/settings/SettingsApp.tsx"
click node_workspace "https://github.com/s-fetah/jarvis/blob/main/apps/desktop/src/renderer/src/components/canvas/WorkspaceCanvas.tsx"
click node_promptcontrols "https://github.com/s-fetah/jarvis/blob/main/apps/desktop/src/renderer/src/components/PromptControlsBar.tsx"
click node_plan "https://github.com/s-fetah/jarvis/blob/main/apps/desktop/src/renderer/src/components/chat/PlanCard.tsx"
click node_audio_ui "https://github.com/s-fetah/jarvis/blob/main/apps/desktop/src/renderer/src/lib/audio-service.ts"
click node_orchestrator "https://github.com/s-fetah/jarvis/blob/main/packages/core/src/agent/agent-orchestrator.ts"
click node_prompt "https://github.com/s-fetah/jarvis/blob/main/packages/core/src/gateway/prompt-engine.ts"
click node_model "https://github.com/s-fetah/jarvis/blob/main/packages/core/src/gateway/model-adapter.ts"
click node_providers "https://github.com/s-fetah/jarvis/tree/main/packages/core/src/gateway/providers"
click node_executor "https://github.com/s-fetah/jarvis/blob/main/packages/core/src/tools/tool-executor.ts"
click node_registry "https://github.com/s-fetah/jarvis/blob/main/packages/core/src/tools/tool-registry.ts"
click node_policy "https://github.com/s-fetah/jarvis/blob/main/packages/core/src/policy/policy-engine.ts"
click node_process "https://github.com/s-fetah/jarvis/blob/main/packages/core/src/process/process-manager.ts"
click node_browser "https://github.com/s-fetah/jarvis/blob/main/packages/core/src/utils/browser.ts"
click node_memory "https://github.com/s-fetah/jarvis/blob/main/packages/core/src/memory/user-memory.ts"
click node_learning "https://github.com/s-fetah/jarvis/blob/main/packages/core/src/memory/learning-engine.ts"
click node_database "https://github.com/s-fetah/jarvis/blob/main/packages/core/src/storage/database.ts"
click node_voice_manager "https://github.com/s-fetah/jarvis/blob/main/packages/voice/src/voice-manager.ts"
click node_vad "https://github.com/s-fetah/jarvis/blob/main/packages/voice/src/vad/speech-boundary-detector.ts"
click node_silero "https://github.com/s-fetah/jarvis/blob/main/packages/voice/src/providers/vad/silero-vad-provider.ts"
click node_tts "https://github.com/s-fetah/jarvis/blob/main/packages/voice/src/providers/tts/tts-dispatcher.ts"
click node_audio_cache "https://github.com/s-fetah/jarvis/blob/main/packages/voice/src/cache/audio-cache-manager.ts"
click node_protocol "https://github.com/s-fetah/jarvis/blob/main/packages/protocol/src/types.ts"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_app,node_quickchat,node_mascot,node_settings,node_workspace,node_promptcontrols,node_plan,node_audio_ui,node_user toneBlue
class node_orchestrator,node_prompt,node_model,node_providers,node_protocol toneAmber
class node_executor,node_registry,node_policy,node_process,node_browser toneMint
class node_voice_manager,node_vad,node_silero,node_tts,node_audio_cache toneRose
class node_memory,node_learning,node_database toneIndigo
```

## Components

### Desktop surfaces

- [Desktop chat shell](https://github.com/s-fetah/jarvis/blob/main/apps/desktop/src/renderer/src/App.tsx): `apps/desktop/src/renderer/src/App.tsx`
- [Quick chat](https://github.com/s-fetah/jarvis/blob/main/apps/desktop/src/renderer/src/quickchat/QuickChatApp.tsx): `apps/desktop/src/renderer/src/quickchat/QuickChatApp.tsx`
- [Mascot window](https://github.com/s-fetah/jarvis/blob/main/apps/desktop/src/main/windows/mascot-manager.ts): `apps/desktop/src/main/windows/mascot-manager.ts`
- [Settings surface](https://github.com/s-fetah/jarvis/blob/main/apps/desktop/src/renderer/src/settings/SettingsApp.tsx): `apps/desktop/src/renderer/src/settings/SettingsApp.tsx`
- [Workspace canvas](https://github.com/s-fetah/jarvis/blob/main/apps/desktop/src/renderer/src/components/canvas/WorkspaceCanvas.tsx): `apps/desktop/src/renderer/src/components/canvas/WorkspaceCanvas.tsx`
- [Prompt controls](https://github.com/s-fetah/jarvis/blob/main/apps/desktop/src/renderer/src/components/PromptControlsBar.tsx): `apps/desktop/src/renderer/src/components/PromptControlsBar.tsx`
- [Plan card](https://github.com/s-fetah/jarvis/blob/main/apps/desktop/src/renderer/src/components/chat/PlanCard.tsx): `apps/desktop/src/renderer/src/components/chat/PlanCard.tsx`
- [Audio capture](https://github.com/s-fetah/jarvis/blob/main/apps/desktop/src/renderer/src/lib/audio-service.ts): `apps/desktop/src/renderer/src/lib/audio-service.ts`

### Agent runtime

- [Agent orchestrator](https://github.com/s-fetah/jarvis/blob/main/packages/core/src/agent/agent-orchestrator.ts): `packages/core/src/agent/agent-orchestrator.ts`
- [Prompt engine](https://github.com/s-fetah/jarvis/blob/main/packages/core/src/gateway/prompt-engine.ts): `packages/core/src/gateway/prompt-engine.ts`
- [Model adapter](https://github.com/s-fetah/jarvis/blob/main/packages/core/src/gateway/model-adapter.ts): `packages/core/src/gateway/model-adapter.ts`
- [Model providers](https://github.com/s-fetah/jarvis/tree/main/packages/core/src/gateway/providers): `packages/core/src/gateway/providers`
- [Shared protocol](https://github.com/s-fetah/jarvis/blob/main/packages/protocol/src/types.ts): `packages/protocol/src/types.ts`

### Tools and policy

- [Tool executor](https://github.com/s-fetah/jarvis/blob/main/packages/core/src/tools/tool-executor.ts): `packages/core/src/tools/tool-executor.ts`
- [Tool registry](https://github.com/s-fetah/jarvis/blob/main/packages/core/src/tools/tool-registry.ts): `packages/core/src/tools/tool-registry.ts`
- [Approval policy](https://github.com/s-fetah/jarvis/blob/main/packages/core/src/policy/policy-engine.ts): `packages/core/src/policy/policy-engine.ts`
- [Process manager](https://github.com/s-fetah/jarvis/blob/main/packages/core/src/process/process-manager.ts): `packages/core/src/process/process-manager.ts`
- [Browser automation](https://github.com/s-fetah/jarvis/blob/main/packages/core/src/utils/browser.ts): `packages/core/src/utils/browser.ts`

### Voice services

- [Voice manager](https://github.com/s-fetah/jarvis/blob/main/packages/voice/src/voice-manager.ts): `packages/voice/src/voice-manager.ts`
- [Speech detection](https://github.com/s-fetah/jarvis/blob/main/packages/voice/src/vad/speech-boundary-detector.ts): `packages/voice/src/vad/speech-boundary-detector.ts`
- [Silero VAD](https://github.com/s-fetah/jarvis/blob/main/packages/voice/src/providers/vad/silero-vad-provider.ts): `packages/voice/src/providers/vad/silero-vad-provider.ts`
- [Speech synthesis](https://github.com/s-fetah/jarvis/blob/main/packages/voice/src/providers/tts/tts-dispatcher.ts): `packages/voice/src/providers/tts/tts-dispatcher.ts`
- [Audio cache](https://github.com/s-fetah/jarvis/blob/main/packages/voice/src/cache/audio-cache-manager.ts): `packages/voice/src/cache/audio-cache-manager.ts`

### Persistence and learning

- [User memory](https://github.com/s-fetah/jarvis/blob/main/packages/core/src/memory/user-memory.ts): `packages/core/src/memory/user-memory.ts`
- [Learning engine](https://github.com/s-fetah/jarvis/blob/main/packages/core/src/memory/learning-engine.ts): `packages/core/src/memory/learning-engine.ts`
- [Conversation storage](https://github.com/s-fetah/jarvis/blob/main/packages/core/src/storage/database.ts): `packages/core/src/storage/database.ts`

### Other components

- **User**

## Connections

- User → Desktop chat shell: submits tasks
- Desktop chat shell → Agent orchestrator: requests work
- Agent orchestrator → Model adapter: requests models (evidence: `packages/core/src/agent/agent-orchestrator.ts`)
- Agent orchestrator → Tool executor: invokes tools (evidence: `packages/core/src/agent/agent-orchestrator.ts`)
- Agent orchestrator → Tool registry: consults tools (evidence: `packages/core/src/agent/agent-orchestrator.ts`)
- Agent orchestrator → Prompt engine: builds prompts
- Prompt engine → User memory: loads profile (evidence: `packages/core/src/gateway/prompt-engine.ts`)
- Prompt engine → Shared protocol: checks safety (evidence: `packages/core/src/gateway/prompt-engine.ts`)
- Model adapter → Model providers: selects provider
- Tool executor → Approval policy: checks permissions (evidence: `packages/core/src/tools/tool-executor.ts`)
- Tool executor → Process manager: manages processes (evidence: `packages/core/src/tools/tool-executor.ts`)
- Tool executor → Browser automation: runs browser tasks (evidence: `packages/core/src/tools/tool-executor.ts`)
- Tool executor → Tool registry: uses definitions (evidence: `packages/core/src/tools/tool-executor.ts`)
- Agent orchestrator → Learning engine: learns conventions (evidence: `packages/core/src/agent/agent-orchestrator.ts`)
- Learning engine → User memory: updates notes (evidence: `packages/core/src/memory/learning-engine.ts`)
- Desktop chat shell → Workspace canvas: renders canvas (evidence: `apps/desktop/src/renderer/src/App.tsx`)
- Desktop chat shell → Plan card: renders plans (evidence: `apps/desktop/src/renderer/src/App.tsx`)
- Quick chat → Prompt controls: renders controls (evidence: `apps/desktop/src/renderer/src/quickchat/QuickChatApp.tsx`)
- Prompt controls → Audio capture: uses audio (evidence: `apps/desktop/src/renderer/src/components/chat/Composer.tsx`)
- Settings surface → Audio capture: configures voice (evidence: `apps/desktop/src/renderer/src/settings/SettingsApp.tsx`)
- Audio capture → Voice manager: sends audio
- Voice manager → Speech detection: detects boundaries (evidence: `packages/voice/src/voice-manager.ts`)
- Speech detection → Silero VAD: uses detector (evidence: `packages/voice/src/vad/speech-boundary-detector.ts`)
- Voice manager → Speech synthesis: dispatches synthesis (evidence: `packages/voice/src/voice-manager.ts`)
- Voice manager → Audio cache: caches speech (evidence: `packages/voice/src/voice-manager.ts`)
- Agent orchestrator → Conversation storage: persists history
- Mascot window → Quick chat: opens drawer (evidence: `apps/desktop/src/main/windows/mascot-manager.ts`)
- Desktop chat shell → Shared protocol: uses shared types (evidence: `apps/desktop/src/renderer/src/App.tsx`)
