# vladkepkin/jarvis architecture

> Architecture diagram of the GitHub repository vladkepkin/jarvis, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-10-05). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/vladkepkin/jarvis
- Repository: https://github.com/vladkepkin/jarvis

## Overview

Jarvis is a voice-controlled assistant with a FastAPI management interface. Its main workflow starts audio capture, recognizes speech through a selected STT provider, resolves the resulting text to a skill intent, and executes configured actions. Plugins supply speech recognition and skills; YAML-backed automation uses a shared pipeline engine. The API exposes status, plugin discovery, config editing, and reload operations. Source excerpts show the main runtime and core contracts; some audio, WebSocket, and plugin wiring is only partially evidenced, so those links are omitted or marked unverified.

Main entry points: [`backend/main.py`](https://github.com/vladkepkin/jarvis/blob/dev/backend/main.py) (Application startup), [`backend/app/core/orchestrator.py`](https://github.com/vladkepkin/jarvis/blob/dev/backend/app/core/orchestrator.py) (Speech orchestrator), [`backend/app/utils/BaseConfigSkill.py`](https://github.com/vladkepkin/jarvis/blob/dev/backend/app/utils/BaseConfigSkill.py) (YAML skill base), [`backend/app/api/routes.py`](https://github.com/vladkepkin/jarvis/blob/dev/backend/app/api/routes.py) (Management routes)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_runtime["Voice runtime"]
  node_main["Application startup<br/>[main.py]"]
  node_mic["Microphone capture<br/>[local_mic.py]"]
  node_audio["Audio manager<br/>[manager.py]"]
  node_orchestrator["Speech orchestrator<br/>[orchestrator.py]"]
end

subgraph group_plugins["Plugin system"]
  node_stt["Vosk speech provider<br/>[plugin.py]"]
  node_interfaces["Plugin contracts<br/>[interfaces.py]"]
  node_loader["Plugin loader<br/>[plugin_loader.py]"]
  node_registry["Plugin registry<br/>[plugin_registry.py]"]
  node_skills["Assistant skills"]
end

subgraph group_automation["Intent and actions"]
  node_resolver["Intent resolver<br/>[intent_resolver.py]"]
  node_base_skill["YAML skill base<br/>[BaseConfigSkill.py]"]
  node_automation_skill["Automation skill<br/>[plugin.py]"]
  node_script_skill["Script runner skill<br/>[plugin.py]"]
  node_system_skill["System skill<br/>[plugin.py]"]
  node_pipeline["Action pipeline<br/>[pipeline_engine.py]"]
  node_system_service["System actions<br/>[system.py]"]
end

subgraph group_management["Management API"]
  node_api_server["HTTP and WebSocket server<br/>[server.py]"]
  node_routes["Management routes<br/>[routes.py]"]
  node_websockets["WebSocket interface<br/>[websockets.py]"]
  node_config_files["User YAML configs"]
end

subgraph group_shared["Shared services"]
  node_event_bus["Event bus<br/>[event_bus.py]"]
  node_settings["Settings paths<br/>[config.py]"]
end

node_user(("Voice user"))

node_user -->|"speaks to"| node_mic
node_main -->|"loads plugins"| node_loader
node_main -->|"starts"| node_orchestrator
node_main -->|"starts capture"| node_mic
node_main -->|"creates server"| node_api_server
node_main -->|"registers intents"| node_resolver
node_mic -.->|"submits audio"| node_audio
node_audio -->|"streams chunks"| node_orchestrator
node_orchestrator -->|"processes audio"| node_stt
node_orchestrator -->|"resolves text"| node_resolver
node_orchestrator -->|"emits events"| node_event_bus
node_loader -.->|"registers plugins"| node_registry
node_stt -->|"implements contract"| node_interfaces
node_automation_skill -->|"uses base"| node_base_skill
node_script_skill -->|"uses base"| node_base_skill
node_system_skill -->|"uses base"| node_base_skill
node_base_skill -->|"registers patterns"| node_resolver
node_base_skill -->|"executes actions"| node_pipeline
node_system_skill -->|"uses actions"| node_system_service
node_api_server -->|"mounts routes"| node_routes
node_routes -->|"lists and reloads"| node_registry
node_routes -->|"uses config paths"| node_settings
node_routes -->|"reads and writes"| node_config_files
node_api_server -->|"mounts interface"| node_websockets
node_websockets -.->|"subscribes to events"| node_event_bus
node_websockets -.->|"submits audio"| node_audio

click node_main "https://github.com/vladkepkin/jarvis/blob/dev/backend/main.py"
click node_mic "https://github.com/vladkepkin/jarvis/blob/dev/backend/app/services/audio/local_mic.py"
click node_audio "https://github.com/vladkepkin/jarvis/blob/dev/backend/app/services/audio/manager.py"
click node_orchestrator "https://github.com/vladkepkin/jarvis/blob/dev/backend/app/core/orchestrator.py"
click node_stt "https://github.com/vladkepkin/jarvis/blob/dev/backend/app/plugins/provider_stt_vosk/plugin.py"
click node_interfaces "https://github.com/vladkepkin/jarvis/blob/dev/backend/app/core/interfaces.py"
click node_loader "https://github.com/vladkepkin/jarvis/blob/dev/backend/app/core/plugin_loader.py"
click node_registry "https://github.com/vladkepkin/jarvis/blob/dev/backend/app/core/plugin_registry.py"
click node_skills "https://github.com/vladkepkin/jarvis/tree/dev/backend/app/plugins"
click node_resolver "https://github.com/vladkepkin/jarvis/blob/dev/backend/app/core/intent_resolver.py"
click node_base_skill "https://github.com/vladkepkin/jarvis/blob/dev/backend/app/utils/BaseConfigSkill.py"
click node_automation_skill "https://github.com/vladkepkin/jarvis/blob/dev/backend/app/plugins/skill_automation/plugin.py"
click node_script_skill "https://github.com/vladkepkin/jarvis/blob/dev/backend/app/plugins/skill_script_runner/plugin.py"
click node_system_skill "https://github.com/vladkepkin/jarvis/blob/dev/backend/app/plugins/skill_system/plugin.py"
click node_pipeline "https://github.com/vladkepkin/jarvis/blob/dev/backend/app/core/pipeline_engine.py"
click node_system_service "https://github.com/vladkepkin/jarvis/blob/dev/backend/app/services/system.py"
click node_event_bus "https://github.com/vladkepkin/jarvis/blob/dev/backend/app/core/event_bus.py"
click node_settings "https://github.com/vladkepkin/jarvis/blob/dev/backend/app/core/config.py"
click node_api_server "https://github.com/vladkepkin/jarvis/blob/dev/backend/app/api/server.py"
click node_routes "https://github.com/vladkepkin/jarvis/blob/dev/backend/app/api/routes.py"
click node_websockets "https://github.com/vladkepkin/jarvis/blob/dev/backend/app/api/websockets.py"
click node_config_files "https://github.com/vladkepkin/jarvis/tree/dev/userdata/configs"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_main,node_mic,node_audio,node_orchestrator,node_user toneBlue
class node_stt,node_interfaces,node_loader,node_registry,node_skills toneAmber
class node_resolver,node_base_skill,node_automation_skill,node_script_skill,node_system_skill,node_pipeline,node_system_service toneMint
class node_api_server,node_routes,node_websockets,node_config_files toneRose
class node_event_bus,node_settings toneIndigo
```

## Components

### Voice runtime

- [Application startup](https://github.com/vladkepkin/jarvis/blob/dev/backend/main.py): `backend/main.py`
- [Microphone capture](https://github.com/vladkepkin/jarvis/blob/dev/backend/app/services/audio/local_mic.py): `backend/app/services/audio/local_mic.py`
- [Audio manager](https://github.com/vladkepkin/jarvis/blob/dev/backend/app/services/audio/manager.py): `backend/app/services/audio/manager.py`
- [Speech orchestrator](https://github.com/vladkepkin/jarvis/blob/dev/backend/app/core/orchestrator.py): `backend/app/core/orchestrator.py`

### Plugin system

- [Vosk speech provider](https://github.com/vladkepkin/jarvis/blob/dev/backend/app/plugins/provider_stt_vosk/plugin.py): `backend/app/plugins/provider_stt_vosk/plugin.py`
- [Plugin contracts](https://github.com/vladkepkin/jarvis/blob/dev/backend/app/core/interfaces.py): `backend/app/core/interfaces.py`
- [Plugin loader](https://github.com/vladkepkin/jarvis/blob/dev/backend/app/core/plugin_loader.py): `backend/app/core/plugin_loader.py`
- [Plugin registry](https://github.com/vladkepkin/jarvis/blob/dev/backend/app/core/plugin_registry.py): `backend/app/core/plugin_registry.py`
- [Assistant skills](https://github.com/vladkepkin/jarvis/tree/dev/backend/app/plugins): `backend/app/plugins`

### Intent and actions

- [Intent resolver](https://github.com/vladkepkin/jarvis/blob/dev/backend/app/core/intent_resolver.py): `backend/app/core/intent_resolver.py`
- [YAML skill base](https://github.com/vladkepkin/jarvis/blob/dev/backend/app/utils/BaseConfigSkill.py): `backend/app/utils/BaseConfigSkill.py`
- [Automation skill](https://github.com/vladkepkin/jarvis/blob/dev/backend/app/plugins/skill_automation/plugin.py): `backend/app/plugins/skill_automation/plugin.py`
- [Script runner skill](https://github.com/vladkepkin/jarvis/blob/dev/backend/app/plugins/skill_script_runner/plugin.py): `backend/app/plugins/skill_script_runner/plugin.py`
- [System skill](https://github.com/vladkepkin/jarvis/blob/dev/backend/app/plugins/skill_system/plugin.py): `backend/app/plugins/skill_system/plugin.py`
- [Action pipeline](https://github.com/vladkepkin/jarvis/blob/dev/backend/app/core/pipeline_engine.py): `backend/app/core/pipeline_engine.py`
- [System actions](https://github.com/vladkepkin/jarvis/blob/dev/backend/app/services/system.py): `backend/app/services/system.py`

### Management API

- [HTTP and WebSocket server](https://github.com/vladkepkin/jarvis/blob/dev/backend/app/api/server.py): `backend/app/api/server.py`
- [Management routes](https://github.com/vladkepkin/jarvis/blob/dev/backend/app/api/routes.py): `backend/app/api/routes.py`
- [WebSocket interface](https://github.com/vladkepkin/jarvis/blob/dev/backend/app/api/websockets.py): `backend/app/api/websockets.py`
- [User YAML configs](https://github.com/vladkepkin/jarvis/tree/dev/userdata/configs): `userdata/configs`

### Shared services

- [Event bus](https://github.com/vladkepkin/jarvis/blob/dev/backend/app/core/event_bus.py): `backend/app/core/event_bus.py`
- [Settings paths](https://github.com/vladkepkin/jarvis/blob/dev/backend/app/core/config.py): `backend/app/core/config.py`

### Other components

- **Voice user**

## Connections

- Voice user → Microphone capture: speaks to
- Application startup → Plugin loader: loads plugins (evidence: `backend/main.py`)
- Application startup → Speech orchestrator: starts (evidence: `backend/main.py`)
- Application startup → Microphone capture: starts capture (evidence: `backend/main.py`)
- Application startup → HTTP and WebSocket server: creates server (evidence: `backend/main.py`)
- Application startup → Intent resolver: registers intents (evidence: `backend/main.py`)
- Microphone capture → Audio manager: submits audio (evidence: `backend/app/services/audio/local_mic.py`)
- Audio manager → Speech orchestrator: streams chunks (evidence: `backend/app/core/orchestrator.py`)
- Speech orchestrator → Vosk speech provider: processes audio (evidence: `backend/app/core/orchestrator.py`)
- Speech orchestrator → Intent resolver: resolves text (evidence: `backend/app/core/orchestrator.py`)
- Speech orchestrator → Event bus: emits events (evidence: `backend/app/core/orchestrator.py`)
- Plugin loader → Plugin registry: registers plugins (evidence: `backend/app/core/plugin_loader.py`)
- Vosk speech provider → Plugin contracts: implements contract (evidence: `backend/app/plugins/provider_stt_vosk/plugin.py`)
- Automation skill → YAML skill base: uses base (evidence: `backend/app/plugins/skill_automation/plugin.py`)
- Script runner skill → YAML skill base: uses base (evidence: `backend/app/plugins/skill_script_runner/plugin.py`)
- System skill → YAML skill base: uses base (evidence: `backend/app/plugins/skill_system/plugin.py`)
- YAML skill base → Intent resolver: registers patterns (evidence: `backend/app/utils/BaseConfigSkill.py`)
- YAML skill base → Action pipeline: executes actions (evidence: `backend/app/utils/BaseConfigSkill.py`)
- System skill → System actions: uses actions (evidence: `backend/app/plugins/skill_system/plugin.py`)
- HTTP and WebSocket server → Management routes: mounts routes (evidence: `backend/app/api/server.py`)
- Management routes → Plugin registry: lists and reloads (evidence: `backend/app/api/routes.py`)
- Management routes → Settings paths: uses config paths (evidence: `backend/app/api/routes.py`)
- Management routes → User YAML configs: reads and writes (evidence: `backend/app/api/routes.py`)
- HTTP and WebSocket server → WebSocket interface: mounts interface (evidence: `backend/app/api/server.py`)
- WebSocket interface → Event bus: subscribes to events (evidence: `backend/app/api/websockets.py`)
- WebSocket interface → Audio manager: submits audio (evidence: `backend/app/api/websockets.py`)
