How vladkepkin/jarvis works
Jarvis is a voice-controlled assistant with a FastAPI management interface. Its main workflow starts audio capture, recognizes speech through a selected STT provider, resolves the resulting text to a skill intent, and executes configured actions. Plugins supply speech recognition and skills; YAML-backed automation uses a shared pipeline engine. The API exposes status, plugin discovery, config editing, and reload operations. Source excerpts show the main runtime and core contracts; some audio, WebSocket, and plugin wiring is only partially evidenced, so those links are omitted or marked unverified.
Main entry points: main.py Application startup · orchestrator.py Speech orchestrator · BaseConfigSkill.py YAML skill base · routes.py Management routes
Components 23
Voice runtime
- Application startup
backend/main.py - Microphone capture
backend/app/services/audio/local_mic.py - Audio manager
backend/app/services/audio/manager.py - Speech orchestrator
backend/app/core/orchestrator.py
Plugin system
- Vosk speech provider
backend/app/plugins/provider_stt_vosk/plugin.py - Plugin contracts
backend/app/core/interfaces.py - Plugin loader
backend/app/core/plugin_loader.py - Plugin registry
backend/app/core/plugin_registry.py - Assistant skills
backend/app/plugins
Intent and actions
- Intent resolver
backend/app/core/intent_resolver.py - YAML skill base
backend/app/utils/BaseConfigSkill.py - Automation skill
backend/app/plugins/skill_automation/plugin.py - Script runner skill
backend/app/plugins/skill_script_runner/plugin.py - System skill
backend/app/plugins/skill_system/plugin.py - Action pipeline
backend/app/core/pipeline_engine.py - System actions
backend/app/services/system.py
Management API
- HTTP and WebSocket server
backend/app/api/server.py - Management routes
backend/app/api/routes.py - WebSocket interface
backend/app/api/websockets.py - User YAML configs
userdata/configs
Shared services
- Event bus
backend/app/core/event_bus.py - Settings paths
backend/app/core/config.py
Other components
- Voice user
Connections 26
- Voice user → Microphone capture: speaks to
- Application startup → Plugin loader: loads plugins
backend/main.py - Application startup → Speech orchestrator: starts
backend/main.py - Application startup → Microphone capture: starts capture
backend/main.py - Application startup → HTTP and WebSocket server: creates server
backend/main.py - Application startup → Intent resolver: registers intents
backend/main.py - Microphone capture → Audio manager: submits audio
backend/app/services/audio/local_mic.py - Audio manager → Speech orchestrator: streams chunks
backend/app/core/orchestrator.py - Speech orchestrator → Vosk speech provider: processes audio
backend/app/core/orchestrator.py - Speech orchestrator → Intent resolver: resolves text
backend/app/core/orchestrator.py - Speech orchestrator → Event bus: emits events
backend/app/core/orchestrator.py - Plugin loader → Plugin registry: registers plugins
backend/app/core/plugin_loader.py - Vosk speech provider → Plugin contracts: implements contract
backend/app/plugins/provider_stt_vosk/plugin.py - Automation skill → YAML skill base: uses base
backend/app/plugins/skill_automation/plugin.py - Script runner skill → YAML skill base: uses base
backend/app/plugins/skill_script_runner/plugin.py - System skill → YAML skill base: uses base
backend/app/plugins/skill_system/plugin.py - YAML skill base → Intent resolver: registers patterns
backend/app/utils/BaseConfigSkill.py - YAML skill base → Action pipeline: executes actions
backend/app/utils/BaseConfigSkill.py - System skill → System actions: uses actions
backend/app/plugins/skill_system/plugin.py - HTTP and WebSocket server → Management routes: mounts routes
backend/app/api/server.py - Management routes → Plugin registry: lists and reloads
backend/app/api/routes.py - Management routes → Settings paths: uses config paths
backend/app/api/routes.py - Management routes → User YAML configs: reads and writes
backend/app/api/routes.py - HTTP and WebSocket server → WebSocket interface: mounts interface
backend/app/api/server.py - WebSocket interface → Event bus: subscribes to events
backend/app/api/websockets.py - WebSocket interface → Audio manager: submits audio
backend/app/api/websockets.py