How harsh-life/jarvis works
JARVIS is a self-hosted assistant with an Android client and a server-side task runtime: users submit tasks, a model proposes actions, deterministic policy checks them, tools execute permitted steps, and risky actions wait for confirmation. The repository also includes device sign-in, configurable models, agent authoring, memory and vault, evaluation, reminders, voice, and a browser Workbench. The README documents the main workflow and optional capabilities; most implementation links are not verified by the sampled source, so uncertain connections are omitted.
Main entry points: runtime.py Agent runtime · app.py HTTP API · models.py Persistent storage · MainActivity.kt Android app
Components 26
Client surfaces
- Android app
android/app/src/main/kotlin/com/hypermind/jarvis/MainActivity.kt - Device channel
android/app/src/main/kotlin/com/hypermind/jarvis/channel/ChannelService.kt - Screen perception
android/app/src/main/kotlin/com/hypermind/jarvis/perception/JarvisAccessibilityService.kt - Workbench
web/js/app.js - Android actions
android/app/src/main/kotlin/com/hypermind/jarvis/channel/OperationHandler.kt
Task runtime
- Agent runtime
server/agent/runtime.py - Run context
server/agent/context.py - Model gateway
server/agents/gateway/model_gateway.py - Agent service
server/agents/service.py - Agent compiler
server/agents/compiler.py - Tool execution
server/tools/executor.py - Constrained execution
server/fs/sandbox.py
Identity and policy
- Authorization
server/graph/authorization.py - Capability policy
server/capabilities/policy.py - Device identity
server/auth/sessions.py - Graph lifecycle
server/graph/service.py
Optional capabilities
- Persistent memory
server/memory/provider.py - Knowledge Vault
server/vault/ingest.py - Judge evaluation
server/evaluation/service.py - Reminders
server/scheduler/service.py - Voice providers
server/voice/service.py - Push wake
server/composition/push.py
Server platform
- HTTP API
server/gateway/app.py - Persistent storage
server/storage/models.py
Other components
- User
- Model provider
Connections 29
- User → Android app: submits tasks
README.md - User → Workbench: configures models
README.md - Android app → HTTP API: sends requests
- Workbench → HTTP API: calls API
web/js/app.js - HTTP API → Device identity: authenticates
- HTTP API → Agent runtime: dispatches tasks
- Agent runtime → Run context: uses context
- Agent runtime → Model gateway: requests proposals
README.md - Model gateway → Model provider: queries model
README.md - Agent runtime → Authorization: checks actions
README.md - Agent runtime → Tool execution: executes tools
README.md - Tool execution → Constrained execution: confines work
README.md - Agent runtime → Persistent storage: stores run state
- Agent runtime → Agent service: uses agent definitions
- Agent service → Agent compiler: compiles drafts
server/agents/service.py - Agent service → Persistent storage: persists definitions
server/agents/service.py - Authorization → Capability policy: checks policy
- Graph lifecycle → Persistent storage: updates memberships
server/graph/service.py - Graph lifecycle → Authorization: supports access checks
- Android app → Device channel: starts connection
android/app/src/main/kotlin/com/hypermind/jarvis/channel/ChannelService.kt - Device channel → Android actions: handles operations
- Device channel → Screen perception: uses device state
- Agent runtime → Persistent memory: uses memory
- User → Knowledge Vault: curates knowledge
server/vault/ingest.py - Knowledge Vault → Persistent storage: indexes documents
- Agent runtime → Judge evaluation: evaluated by
- User → Reminders: sets reminders
README.md - User → Voice providers: speaks requests
- User → Push wake: receives wake