How thaisontv123/jarvis works
Jarvis is a Vietnamese, voice-first personal assistant delivered as a Windows desktop app with a local Python API. The main workflow runs from the desktop or chat API through conversation orchestration, model responses, and registered tools; risky actions require permission checks and confirmation before execution. Capability packs cover productivity, web, Google services, and computer control. Separate branches provide persistent memory, scheduled notifications, and user-approved automated routines. The README documents these capabilities; some subsystem connections are shown from documentation because their implementation was not sampled.
Main entry points: orchestrator.py Agent orchestration · routes_chat.py Chat endpoint · registry.py Tool registry · engine.py Scheduler
Components 22
User interfaces
- Desktop shell
desktop/src/main.rs - Desktop UI
app/desktop/static/desktop.js - Local API
app/main.py - Chat endpoint
app/api/routes_chat.py
Conversation and voice
- Voice session
app/desktop/session.py - Speech recognition
app/stt/gemini_live_provider.py - Agent orchestration
app/core/orchestrator.py - AI provider
app/providers/gemini_provider.py - Speech output
app/tts/workflow.py
Capability packs
- Capability packs
app/packs/builtin.py - Computer control
app/packs/computer/service.py - External integrations
app/integrations/runtime.py - Memory retrieval
app/memory/retrieval.py
Safety and state
- Tool registry
app/core/registry.py - Permission engine
app/core/permissions.py - Confirmation
app/core/confirmation.py - Action audit
app/core/audit.py - Application database
app/database/models.py
Proactive automation
- Scheduler
app/scheduler/engine.py - Automated routines
app/routines/engine.py - Observations
app/memory/observations.py
Other components
- User
Connections 24
- User → Desktop shell: uses
- Desktop shell → Desktop UI: opens
- Desktop UI → Local API: calls same-origin
- Local API → Chat endpoint: routes requests
app/main.py - User → Chat endpoint: sends chat
- Chat endpoint → Agent orchestration: invokes
app/api/routes_chat.py - Voice session → Speech recognition: transcribes
app/desktop/session.py - Voice session → Agent orchestration: submits turns
- Agent orchestration → AI provider: requests responses
- Agent orchestration → Tool registry: dispatches tools
app/core/orchestrator.py - Tool registry → Permission engine: checks risk
app/core/registry.py - Agent orchestration → Confirmation: requests approval
app/core/orchestrator.py - Confirmation → Tool registry: executes approved
app/api/routes_confirmations.py - Confirmation → Action audit: records outcome
app/api/routes_confirmations.py - Chat endpoint → Application database: uses sessions
app/api/routes_chat.py - Tool registry → Capability packs: registers capabilities
app/packs/builtin.py - Capability packs → Computer control: includes computer
app/packs/builtin.py - Capability packs → External integrations: uses service clients
app/integrations/runtime.py - Agent orchestration → Memory retrieval: retrieves context
- Scheduler → Application database: reads and writes
app/scheduler/engine.py - Scheduler → Automated routines: runs schedules
app/scheduler/runtime.py - Scheduler → Speech output: delivers speech
- Observations → Application database: stores observations
- User → Voice session: speaks