How s-fetah/jarvis works
Jarvis is an experimental local-first Windows desktop assistant. The main workflow is user chat through the desktop UI to the agent orchestrator, which plans tasks, requests model responses, invokes tools, and can seek approval; results and progress return to the UI. The repository also includes mascot and quick-chat surfaces, workspace previews, persistent memory, and voice input/output. The README documents several capabilities whose runtime wiring is not shown in the excerpts, so those links are omitted or marked unverified rather than inferred.
Main entry points: agent-orchestrator.ts Agent orchestrator · App.tsx Desktop chat shell · tool-executor.ts Tool executor · voice-manager.ts Voice manager
Components 27
Desktop surfaces
- Desktop chat shell
apps/desktop/src/renderer/src/App.tsx - Quick chat
apps/desktop/src/renderer/src/quickchat/QuickChatApp.tsx - Mascot window
apps/desktop/src/main/windows/mascot-manager.ts - Settings surface
apps/desktop/src/renderer/src/settings/SettingsApp.tsx - Workspace canvas
apps/desktop/src/renderer/src/components/canvas/WorkspaceCanvas.tsx - Prompt controls
apps/desktop/src/renderer/src/components/PromptControlsBar.tsx - Plan card
apps/desktop/src/renderer/src/components/chat/PlanCard.tsx - Audio capture
apps/desktop/src/renderer/src/lib/audio-service.ts
Agent runtime
- Agent orchestrator
packages/core/src/agent/agent-orchestrator.ts - Prompt engine
packages/core/src/gateway/prompt-engine.ts - Model adapter
packages/core/src/gateway/model-adapter.ts - Model providers
packages/core/src/gateway/providers - Shared protocol
packages/protocol/src/types.ts
Tools and policy
- Tool executor
packages/core/src/tools/tool-executor.ts - Tool registry
packages/core/src/tools/tool-registry.ts - Approval policy
packages/core/src/policy/policy-engine.ts - Process manager
packages/core/src/process/process-manager.ts - Browser automation
packages/core/src/utils/browser.ts
Voice services
- Voice manager
packages/voice/src/voice-manager.ts - Speech detection
packages/voice/src/vad/speech-boundary-detector.ts - Silero VAD
packages/voice/src/providers/vad/silero-vad-provider.ts - Speech synthesis
packages/voice/src/providers/tts/tts-dispatcher.ts - Audio cache
packages/voice/src/cache/audio-cache-manager.ts
Persistence and learning
- User memory
packages/core/src/memory/user-memory.ts - Learning engine
packages/core/src/memory/learning-engine.ts - Conversation storage
packages/core/src/storage/database.ts
Other components
- User
Connections 28
- User → Desktop chat shell: submits tasks
- Desktop chat shell → Agent orchestrator: requests work
- Agent orchestrator → Model adapter: requests models
packages/core/src/agent/agent-orchestrator.ts - Agent orchestrator → Tool executor: invokes tools
packages/core/src/agent/agent-orchestrator.ts - Agent orchestrator → Tool registry: consults tools
packages/core/src/agent/agent-orchestrator.ts - Agent orchestrator → Prompt engine: builds prompts
- Prompt engine → User memory: loads profile
packages/core/src/gateway/prompt-engine.ts - Prompt engine → Shared protocol: checks safety
packages/core/src/gateway/prompt-engine.ts - Model adapter → Model providers: selects provider
- Tool executor → Approval policy: checks permissions
packages/core/src/tools/tool-executor.ts - Tool executor → Process manager: manages processes
packages/core/src/tools/tool-executor.ts - Tool executor → Browser automation: runs browser tasks
packages/core/src/tools/tool-executor.ts - Tool executor → Tool registry: uses definitions
packages/core/src/tools/tool-executor.ts - Agent orchestrator → Learning engine: learns conventions
packages/core/src/agent/agent-orchestrator.ts - Learning engine → User memory: updates notes
packages/core/src/memory/learning-engine.ts - Desktop chat shell → Workspace canvas: renders canvas
apps/desktop/src/renderer/src/App.tsx - Desktop chat shell → Plan card: renders plans
apps/desktop/src/renderer/src/App.tsx - Quick chat → Prompt controls: renders controls
apps/desktop/src/renderer/src/quickchat/QuickChatApp.tsx - Prompt controls → Audio capture: uses audio
apps/desktop/src/renderer/src/components/chat/Composer.tsx - Settings surface → Audio capture: configures voice
apps/desktop/src/renderer/src/settings/SettingsApp.tsx - Audio capture → Voice manager: sends audio
- Voice manager → Speech detection: detects boundaries
packages/voice/src/voice-manager.ts - Speech detection → Silero VAD: uses detector
packages/voice/src/vad/speech-boundary-detector.ts - Voice manager → Speech synthesis: dispatches synthesis
packages/voice/src/voice-manager.ts - Voice manager → Audio cache: caches speech
packages/voice/src/voice-manager.ts - Agent orchestrator → Conversation storage: persists history
- Mascot window → Quick chat: opens drawer
apps/desktop/src/main/windows/mascot-manager.ts - Desktop chat shell → Shared protocol: uses shared types
apps/desktop/src/renderer/src/App.tsx