# hudson191-max/j.a.r.v.i.s architecture

> Architecture diagram of the GitHub repository hudson191-max/j.a.r.v.i.s, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-11-13). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/hudson191-max/j.a.r.v.i.s
- Repository: https://github.com/hudson191-max/j.a.r.v.i.s

## Overview

J.A.R.V.I.S. is an offline-first Windows voice-assistant shell with optional network-backed model and speech services. Its main interaction captures speech through wake-word or push-to-talk input, transcribes it, processes a conversation and tool calls through Hermes, then synthesizes and plays a spoken reply. Desktop controls, dashboard diagnostics, debug-capture replay, and computer/screen features are supporting branches. Relationships are limited to those shown in excerpts, the source index, or README; some unsampled integrations are left unconnected.

Main entry points: [`src/jarvis/engine/controller.py`](https://github.com/hudson191-max/j.a.r.v.i.s/blob/main/src/jarvis/engine/controller.py) (Conversation controller), [`main.py`](https://github.com/hudson191-max/j.a.r.v.i.s/blob/main/main.py) (App entry), [`src/jarvis/hermes/agent.py`](https://github.com/hudson191-max/j.a.r.v.i.s/blob/main/src/jarvis/hermes/agent.py) (Hermes agent), [`src/jarvis/hermes/tools.py`](https://github.com/hudson191-max/j.a.r.v.i.s/blob/main/src/jarvis/hermes/tools.py) (Tool registry)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_runtime["Assistant runtime"]
  node_entry["App entry<br/>[main.py]"]
  node_controller["Conversation controller<br/>[controller.py]"]
  node_contracts["Event contracts<br/>[contracts.py]"]
end

subgraph group_audio["Voice pipeline"]
  node_audioinput["Audio capture<br/>[input.py]"]
  node_wake["Wake detection<br/>[wake.py]"]
  node_transcribe["Whisper transcription<br/>[whisper.py]"]
  node_tts["Speech synthesis<br/>[base.py]"]
  node_playback["Audio playback<br/>[playback.py]"]
end

subgraph group_hermes["Hermes intelligence"]
  node_agent["Hermes agent<br/>[agent.py]"]
  node_client["Model client<br/>[client.py]"]
  node_tools["Tool registry<br/>[tools.py]"]
  node_builtin["Built-in capabilities<br/>[builtin_tools.py]"]
  node_delegates["Delegated agents<br/>[delegate_tools.py]"]
end

subgraph group_desktop["Desktop experience"]
  node_overlay["Assistant capsule<br/>[overlay.py]"]
  node_tray["System tray<br/>[tray.py]"]
  node_hotkey["Push-to-talk key<br/>[hotkey.py]"]
  node_dashboard["Dashboard server<br/>[server.py]"]
end

subgraph group_support["Settings and diagnostics"]
  node_debug[("Debug captures<br/>[debug.py]")]
  node_config["Runtime configuration<br/>[config.py]"]
  node_settings["Assistant settings"]
  node_computer["Computer and screen<br/>[computer_use.py]"]
end

node_user(("User"))

node_user -->|"speaks to"| node_audioinput
node_user -->|"presses"| node_hotkey
node_entry -->|"starts"| node_controller
node_entry -->|"creates"| node_overlay
node_entry -->|"creates"| node_tray
node_entry -->|"configures"| node_hotkey
node_controller -.->|"captures from"| node_audioinput
node_controller -->|"uses"| node_wake
node_controller -->|"transcribes with"| node_transcribe
node_controller -.->|"processes turns with"| node_agent
node_controller -->|"synthesizes replies with"| node_tts
node_controller -.->|"plays replies through"| node_playback
node_agent -.->|"requests model events"| node_client
node_agent -.->|"dispatches tools"| node_tools
node_builtin -->|"registers capabilities"| node_tools
node_delegates -->|"uses registry"| node_tools
node_controller -->|"records captures"| node_debug
node_dashboard -->|"lists and replays"| node_debug
node_dashboard -->|"replays requests with"| node_agent
node_dashboard -->|"synthesizes preview via"| node_tts
node_controller -->|"uses computer control"| node_computer
node_controller -->|"configures session features"| node_settings
node_settings -->|"registers settings tools"| node_tools
node_controller -.->|"emits assistant state"| node_overlay
node_config -->|"configures runtime"| node_controller
node_contracts -->|"defines transcript type"| node_transcribe

click node_entry "https://github.com/hudson191-max/j.a.r.v.i.s/blob/main/main.py"
click node_controller "https://github.com/hudson191-max/j.a.r.v.i.s/blob/main/src/jarvis/engine/controller.py"
click node_contracts "https://github.com/hudson191-max/j.a.r.v.i.s/blob/main/src/jarvis/contracts.py"
click node_audioinput "https://github.com/hudson191-max/j.a.r.v.i.s/blob/main/src/jarvis/audio/input.py"
click node_wake "https://github.com/hudson191-max/j.a.r.v.i.s/blob/main/src/jarvis/audio/wake.py"
click node_transcribe "https://github.com/hudson191-max/j.a.r.v.i.s/blob/main/src/jarvis/transcription/whisper.py"
click node_tts "https://github.com/hudson191-max/j.a.r.v.i.s/blob/main/src/jarvis/tts/base.py"
click node_playback "https://github.com/hudson191-max/j.a.r.v.i.s/blob/main/src/jarvis/audio/playback.py"
click node_agent "https://github.com/hudson191-max/j.a.r.v.i.s/blob/main/src/jarvis/hermes/agent.py"
click node_client "https://github.com/hudson191-max/j.a.r.v.i.s/blob/main/src/jarvis/hermes/client.py"
click node_tools "https://github.com/hudson191-max/j.a.r.v.i.s/blob/main/src/jarvis/hermes/tools.py"
click node_builtin "https://github.com/hudson191-max/j.a.r.v.i.s/blob/main/src/jarvis/hermes/builtin_tools.py"
click node_delegates "https://github.com/hudson191-max/j.a.r.v.i.s/blob/main/src/jarvis/hermes/delegate_tools.py"
click node_overlay "https://github.com/hudson191-max/j.a.r.v.i.s/blob/main/src/jarvis/ui/overlay.py"
click node_tray "https://github.com/hudson191-max/j.a.r.v.i.s/blob/main/src/jarvis/ui/tray.py"
click node_hotkey "https://github.com/hudson191-max/j.a.r.v.i.s/blob/main/src/jarvis/ui/hotkey.py"
click node_dashboard "https://github.com/hudson191-max/j.a.r.v.i.s/blob/main/src/jarvis/dashboard/server.py"
click node_debug "https://github.com/hudson191-max/j.a.r.v.i.s/blob/main/src/jarvis/debug.py"
click node_config "https://github.com/hudson191-max/j.a.r.v.i.s/blob/main/src/jarvis/config.py"
click node_settings "https://github.com/hudson191-max/j.a.r.v.i.s/blob/main/src/jarvis/platform/assistant_settings.py"
click node_computer "https://github.com/hudson191-max/j.a.r.v.i.s/blob/main/src/jarvis/platform/computer_use.py"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_entry,node_controller,node_contracts,node_user toneBlue
class node_audioinput,node_wake,node_transcribe,node_tts,node_playback toneAmber
class node_agent,node_client,node_tools,node_builtin,node_delegates toneMint
class node_overlay,node_tray,node_hotkey,node_dashboard toneRose
class node_debug,node_config,node_settings,node_computer toneIndigo
```

## Components

### Assistant runtime

- [App entry](https://github.com/hudson191-max/j.a.r.v.i.s/blob/main/main.py): `main.py`
- [Conversation controller](https://github.com/hudson191-max/j.a.r.v.i.s/blob/main/src/jarvis/engine/controller.py): `src/jarvis/engine/controller.py`
- [Event contracts](https://github.com/hudson191-max/j.a.r.v.i.s/blob/main/src/jarvis/contracts.py): `src/jarvis/contracts.py`

### Voice pipeline

- [Audio capture](https://github.com/hudson191-max/j.a.r.v.i.s/blob/main/src/jarvis/audio/input.py): `src/jarvis/audio/input.py`
- [Wake detection](https://github.com/hudson191-max/j.a.r.v.i.s/blob/main/src/jarvis/audio/wake.py): `src/jarvis/audio/wake.py`
- [Whisper transcription](https://github.com/hudson191-max/j.a.r.v.i.s/blob/main/src/jarvis/transcription/whisper.py): `src/jarvis/transcription/whisper.py`
- [Speech synthesis](https://github.com/hudson191-max/j.a.r.v.i.s/blob/main/src/jarvis/tts/base.py): `src/jarvis/tts/base.py`
- [Audio playback](https://github.com/hudson191-max/j.a.r.v.i.s/blob/main/src/jarvis/audio/playback.py): `src/jarvis/audio/playback.py`

### Hermes intelligence

- [Hermes agent](https://github.com/hudson191-max/j.a.r.v.i.s/blob/main/src/jarvis/hermes/agent.py): `src/jarvis/hermes/agent.py`
- [Model client](https://github.com/hudson191-max/j.a.r.v.i.s/blob/main/src/jarvis/hermes/client.py): `src/jarvis/hermes/client.py`
- [Tool registry](https://github.com/hudson191-max/j.a.r.v.i.s/blob/main/src/jarvis/hermes/tools.py): `src/jarvis/hermes/tools.py`
- [Built-in capabilities](https://github.com/hudson191-max/j.a.r.v.i.s/blob/main/src/jarvis/hermes/builtin_tools.py): `src/jarvis/hermes/builtin_tools.py`
- [Delegated agents](https://github.com/hudson191-max/j.a.r.v.i.s/blob/main/src/jarvis/hermes/delegate_tools.py): `src/jarvis/hermes/delegate_tools.py`

### Desktop experience

- [Assistant capsule](https://github.com/hudson191-max/j.a.r.v.i.s/blob/main/src/jarvis/ui/overlay.py): `src/jarvis/ui/overlay.py`
- [System tray](https://github.com/hudson191-max/j.a.r.v.i.s/blob/main/src/jarvis/ui/tray.py): `src/jarvis/ui/tray.py`
- [Push-to-talk key](https://github.com/hudson191-max/j.a.r.v.i.s/blob/main/src/jarvis/ui/hotkey.py): `src/jarvis/ui/hotkey.py`
- [Dashboard server](https://github.com/hudson191-max/j.a.r.v.i.s/blob/main/src/jarvis/dashboard/server.py): `src/jarvis/dashboard/server.py`

### Settings and diagnostics

- [Debug captures](https://github.com/hudson191-max/j.a.r.v.i.s/blob/main/src/jarvis/debug.py): `src/jarvis/debug.py`
- [Runtime configuration](https://github.com/hudson191-max/j.a.r.v.i.s/blob/main/src/jarvis/config.py): `src/jarvis/config.py`
- [Assistant settings](https://github.com/hudson191-max/j.a.r.v.i.s/blob/main/src/jarvis/platform/assistant_settings.py): `src/jarvis/platform/assistant_settings.py`
- [Computer and screen](https://github.com/hudson191-max/j.a.r.v.i.s/blob/main/src/jarvis/platform/computer_use.py): `src/jarvis/platform/computer_use.py`

### Other components

- **User**

## Connections

- User → Audio capture: speaks to
- User → Push-to-talk key: presses
- App entry → Conversation controller: starts (evidence: `main.py`)
- App entry → Assistant capsule: creates (evidence: `main.py`)
- App entry → System tray: creates (evidence: `main.py`)
- App entry → Push-to-talk key: configures (evidence: `main.py`)
- Conversation controller → Audio capture: captures from (evidence: `src/jarvis/engine/controller.py`)
- Conversation controller → Wake detection: uses (evidence: `src/jarvis/engine/controller.py`)
- Conversation controller → Whisper transcription: transcribes with (evidence: `src/jarvis/engine/controller.py`)
- Conversation controller → Hermes agent: processes turns with
- Conversation controller → Speech synthesis: synthesizes replies with (evidence: `src/jarvis/engine/controller.py`)
- Conversation controller → Audio playback: plays replies through (evidence: `src/jarvis/engine/controller.py`)
- Hermes agent → Model client: requests model events
- Hermes agent → Tool registry: dispatches tools
- Built-in capabilities → Tool registry: registers capabilities (evidence: `src/jarvis/hermes/builtin_tools.py`)
- Delegated agents → Tool registry: uses registry (evidence: `src/jarvis/hermes/delegate_tools.py`)
- Conversation controller → Debug captures: records captures (evidence: `src/jarvis/engine/controller.py`)
- Dashboard server → Debug captures: lists and replays (evidence: `src/jarvis/dashboard/server.py`)
- Dashboard server → Hermes agent: replays requests with (evidence: `src/jarvis/dashboard/server.py`)
- Dashboard server → Speech synthesis: synthesizes preview via (evidence: `src/jarvis/dashboard/server.py`)
- Conversation controller → Computer and screen: uses computer control (evidence: `src/jarvis/engine/controller.py`)
- Conversation controller → Assistant settings: configures session features (evidence: `src/jarvis/engine/controller.py`)
- Assistant settings → Tool registry: registers settings tools (evidence: `src/jarvis/platform/assistant_settings.py`)
- Conversation controller → Assistant capsule: emits assistant state
- Runtime configuration → Conversation controller: configures runtime (evidence: `src/jarvis/engine/controller.py`)
- Event contracts → Whisper transcription: defines transcript type (evidence: `src/jarvis/transcription/whisper.py`)