How eadmin2/jarvis_ai works
J.A.R.V.I.S is a self-hosted voice assistant and browser HUD backed by Hermes Agent. The main workflow is microphone audio from the browser or push-to-talk client into the server, transcription, a persistent Hermes conversation, then streamed speech and HUD updates. The map also preserves documented approval, interruption, usage, dashboard, media-panel, and worker-monitoring capabilities. Server internals are represented as responsibilities of the same source file; unsampled implementation details are not inferred.
Main entry points: server.py Voice session pipeline · index.html Browser HUD · __init__.py HUD tool registration · client.py Push-to-talk client
Components 20
Voice & HUD
- Browser HUD
server/hud/index.html - Push-to-talk client
client/client.py
Voice Server
- Voice session pipeline
server/server.py - Speech transcription
server/server.py - Hermes API client
server/server.py - Auth & approvals
server/server.py - Streaming speech output
server/server.py - TTS privacy filter
server/server.py - Usage tracking
server/server.py - HUD API & events
server/server.py - Dashboard proxy
server/server.py
Agent & Integrations
- HUD tool registration
hermes-plugin/hud_display/__init__.py - Display tool schemas
hermes-plugin/hud_display/schemas.py - Display tool handlers
hermes-plugin/hud_display/tools.py - Hermes Agent
- ElevenLabs TTS
- Hermes dashboard
Worker Services
- GPU transcription service
worker/stt_server.py - Worker stats service
worker/worker_stats.py
Other components
- User
Connections 21
- User → Browser HUD: speaks or types
README.md - Browser HUD → Voice session pipeline: streams audio
README.md - User → Push-to-talk client: speaks
README.md - Push-to-talk client → Voice session pipeline: streams audio
client/client.py - Voice session pipeline → Speech transcription: transcribes audio
server/server.py - Speech transcription → GPU transcription service: tries remote STT
server/server.py - Voice session pipeline → Hermes API client: dispatches turn
server/server.py - Hermes API client → Hermes Agent: calls API
README.md - Voice session pipeline → Auth & approvals: gates requests
server/server.py - Voice session pipeline → TTS privacy filter: redacts TTS text
README.md - Voice session pipeline → Streaming speech output: streams reply audio
README.md - Streaming speech output → ElevenLabs TTS: requests speech
README.md - Voice session pipeline → Usage tracking: records usage
server/server.py - Browser HUD → HUD API & events: sends HUD actions
README.md - HUD tool registration → Display tool schemas: registers schemas
hermes-plugin/hud_display/__init__.py - HUD tool registration → Display tool handlers: registers handlers
hermes-plugin/hud_display/__init__.py - Display tool handlers → HUD API & events: posts panel actions
hermes-plugin/hud_display/tools.py - Hermes Agent → HUD tool registration: loads tools
- HUD API & events → Dashboard proxy: proxies dashboards
README.md - Dashboard proxy → Hermes dashboard: proxies requests
server/server.py - Voice session pipeline → Worker stats service: fetches worker stats
README.md