# cqink/johnyassist architecture

> Architecture diagram of the GitHub repository cqink/johnyassist, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-09-26). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/cqink/johnyassist
- Repository: https://github.com/cqink/johnyassist
- Video tour (about a minute): https://gitdiagram.com/cqink/johnyassist/video

## Overview

Johnny is a Windows voice assistant: a user speaks a wake-word command, speech is recognized and routed locally or through an optional language model, then an action or scenario runs and the result is spoken and recorded. The graph includes documented image/OCR, face, translation, memory, browser, and plugin capabilities. Some subsystem wiring is inferred only where explicit imports or excerpts support it; unsampled implementations are represented by their exact tree paths, with uncertain links omitted.

Main entry points: [`johnny/controller.py`](https://github.com/cqink/johnyassist/blob/main/johnny/controller.py) (Listening controller), [`johnny/app.py`](https://github.com/cqink/johnyassist/blob/main/johnny/app.py) (Command pipeline), [`johnny/actions/registry.py`](https://github.com/cqink/johnyassist/blob/main/johnny/actions/registry.py) (Action dispatcher), [`johnny/tts_local.py`](https://github.com/cqink/johnyassist/blob/main/johnny/tts_local.py) (TTS backends)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_voice["Voice interaction"]
  node_entry["Application entry<br/>[main.py]"]
  node_controller["Listening controller<br/>[controller.py]"]
  node_listener["Wake-word listener<br/>[listener.py]"]
  node_recognizer["Speech recognition<br/>[recognizer.py]"]
  node_audio["Audio capture<br/>[audio.py]"]
  node_speaker["Speech output<br/>[speaker.py]"]
  node_tts["TTS backends<br/>[tts_local.py]"]
end

subgraph group_command["Command handling"]
  node_app["Command pipeline<br/>[app.py]"]
  node_router["Command router<br/>[router.py]"]
  node_brain["Language-model providers<br/>[brain.py]"]
  node_chain["Scenario runner<br/>[chain.py]"]
end

subgraph group_actions["Actions and integrations"]
  node_actions["Action dispatcher<br/>[registry.py]"]
  node_browser["Browser control<br/>[browser.py]"]
  node_system["System actions<br/>[system.py]"]
  node_connector_action["Connector actions"]
  node_connectors["Vision and utility connectors"]
  node_plugins["Action plugins<br/>[plugins.py]"]
end

subgraph group_state["Assistant state"]
  node_history[("Conversation history<br/>[history.py]")]
  node_memory[("Assistant memory<br/>[memory.py]")]
  node_config["Configuration<br/>[config.py]"]
end

subgraph group_interfaces["User interfaces"]
  node_panel["Settings panel<br/>[panel.py]"]
  node_tray["System tray<br/>[tray.py]"]
end

node_user(("User"))
node_os["Windows and apps"]
node_cloud["Model and connector services"]
node_tts_service["Optional TTS service<br/>[tts_server.py]"]

node_user -->|"speaks into"| node_audio
node_entry -->|"starts"| node_app
node_app -->|"runs"| node_controller
node_controller -->|"waits for wake word"| node_listener
node_controller -->|"recognizes speech"| node_recognizer
node_controller -->|"captures audio"| node_audio
node_controller -->|"submits command"| node_app
node_app -->|"matches command"| node_router
node_app -->|"interprets fallback"| node_brain
node_app -->|"runs scenario"| node_chain
node_app -->|"dispatches action"| node_actions
node_actions -->|"dispatches"| node_browser
node_actions -->|"dispatches"| node_system
node_actions -->|"dispatches"| node_connector_action
node_connector_action -->|"builds and calls"| node_connectors
node_actions -.->|"loads extensions"| node_plugins
node_controller -->|"speaks outcome"| node_speaker
node_speaker -->|"synthesizes speech"| node_tts
node_tts -.->|"requests voice"| node_cloud
node_tts -.->|"requests synthesis"| node_tts_service
node_controller -->|"records turns"| node_history
node_app -->|"uses memory actions"| node_memory
node_app -->|"reads settings"| node_config
node_controller -->|"updates activity"| node_panel
node_user -->|"controls assistant"| node_panel
node_panel -->|"toggles pause"| node_controller
node_tray -->|"controls assistant"| node_controller
node_browser -->|"controls browser"| node_os
node_system -->|"controls system"| node_os
node_brain -.->|"requests interpretation"| node_cloud
node_connectors -.->|"requests services"| node_cloud

click node_entry "https://github.com/cqink/johnyassist/blob/main/main.py"
click node_controller "https://github.com/cqink/johnyassist/blob/main/johnny/controller.py"
click node_listener "https://github.com/cqink/johnyassist/blob/main/johnny/listener.py"
click node_recognizer "https://github.com/cqink/johnyassist/blob/main/johnny/recognizer.py"
click node_audio "https://github.com/cqink/johnyassist/blob/main/johnny/audio.py"
click node_app "https://github.com/cqink/johnyassist/blob/main/johnny/app.py"
click node_router "https://github.com/cqink/johnyassist/blob/main/johnny/router.py"
click node_brain "https://github.com/cqink/johnyassist/blob/main/johnny/brain.py"
click node_chain "https://github.com/cqink/johnyassist/blob/main/johnny/chain.py"
click node_actions "https://github.com/cqink/johnyassist/blob/main/johnny/actions/registry.py"
click node_browser "https://github.com/cqink/johnyassist/blob/main/johnny/actions/browser.py"
click node_system "https://github.com/cqink/johnyassist/blob/main/johnny/actions/system.py"
click node_connector_action "https://github.com/cqink/johnyassist/blob/main/johnny/actions/connector_action.py"
click node_connectors "https://github.com/cqink/johnyassist/tree/main/johnny/connectors"
click node_plugins "https://github.com/cqink/johnyassist/blob/main/johnny/plugins.py"
click node_speaker "https://github.com/cqink/johnyassist/blob/main/johnny/speaker.py"
click node_tts "https://github.com/cqink/johnyassist/blob/main/johnny/tts_local.py"
click node_history "https://github.com/cqink/johnyassist/blob/main/johnny/history.py"
click node_memory "https://github.com/cqink/johnyassist/blob/main/johnny/memory.py"
click node_config "https://github.com/cqink/johnyassist/blob/main/johnny/config.py"
click node_panel "https://github.com/cqink/johnyassist/blob/main/johnny/panel.py"
click node_tray "https://github.com/cqink/johnyassist/blob/main/johnny/tray.py"
click node_tts_service "https://github.com/cqink/johnyassist/blob/main/services/tts-server/tts_server.py"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_entry,node_controller,node_listener,node_recognizer,node_audio,node_speaker,node_tts,node_user toneBlue
class node_app,node_router,node_brain,node_chain toneAmber
class node_actions,node_browser,node_system,node_connector_action,node_connectors,node_plugins toneMint
class node_history,node_memory,node_config toneRose
class node_panel,node_tray,node_os,node_cloud toneIndigo
class node_tts_service toneTeal
```

## Components

### Voice interaction

- [Application entry](https://github.com/cqink/johnyassist/blob/main/main.py): `main.py`
- [Listening controller](https://github.com/cqink/johnyassist/blob/main/johnny/controller.py): `johnny/controller.py`
- [Wake-word listener](https://github.com/cqink/johnyassist/blob/main/johnny/listener.py): `johnny/listener.py`
- [Speech recognition](https://github.com/cqink/johnyassist/blob/main/johnny/recognizer.py): `johnny/recognizer.py`
- [Audio capture](https://github.com/cqink/johnyassist/blob/main/johnny/audio.py): `johnny/audio.py`
- [Speech output](https://github.com/cqink/johnyassist/blob/main/johnny/speaker.py): `johnny/speaker.py`
- [TTS backends](https://github.com/cqink/johnyassist/blob/main/johnny/tts_local.py): `johnny/tts_local.py`

### Command handling

- [Command pipeline](https://github.com/cqink/johnyassist/blob/main/johnny/app.py): `johnny/app.py`
- [Command router](https://github.com/cqink/johnyassist/blob/main/johnny/router.py): `johnny/router.py`
- [Language-model providers](https://github.com/cqink/johnyassist/blob/main/johnny/brain.py): `johnny/brain.py`
- [Scenario runner](https://github.com/cqink/johnyassist/blob/main/johnny/chain.py): `johnny/chain.py`

### Actions and integrations

- [Action dispatcher](https://github.com/cqink/johnyassist/blob/main/johnny/actions/registry.py): `johnny/actions/registry.py`
- [Browser control](https://github.com/cqink/johnyassist/blob/main/johnny/actions/browser.py): `johnny/actions/browser.py`
- [System actions](https://github.com/cqink/johnyassist/blob/main/johnny/actions/system.py): `johnny/actions/system.py`
- [Connector actions](https://github.com/cqink/johnyassist/blob/main/johnny/actions/connector_action.py): `johnny/actions/connector_action.py`
- [Vision and utility connectors](https://github.com/cqink/johnyassist/tree/main/johnny/connectors): `johnny/connectors`
- [Action plugins](https://github.com/cqink/johnyassist/blob/main/johnny/plugins.py): `johnny/plugins.py`

### Assistant state

- [Conversation history](https://github.com/cqink/johnyassist/blob/main/johnny/history.py): `johnny/history.py`
- [Assistant memory](https://github.com/cqink/johnyassist/blob/main/johnny/memory.py): `johnny/memory.py`
- [Configuration](https://github.com/cqink/johnyassist/blob/main/johnny/config.py): `johnny/config.py`

### User interfaces

- [Settings panel](https://github.com/cqink/johnyassist/blob/main/johnny/panel.py): `johnny/panel.py`
- [System tray](https://github.com/cqink/johnyassist/blob/main/johnny/tray.py): `johnny/tray.py`

### Other components

- **User**
- **Windows and apps**
- **Model and connector services**
- [Optional TTS service](https://github.com/cqink/johnyassist/blob/main/services/tts-server/tts_server.py): `services/tts-server/tts_server.py`

## Connections

- User → Audio capture: speaks into
- Application entry → Command pipeline: starts
- Command pipeline → Listening controller: runs
- Listening controller → Wake-word listener: waits for wake word
- Listening controller → Speech recognition: recognizes speech
- Listening controller → Audio capture: captures audio
- Listening controller → Command pipeline: submits command
- Command pipeline → Command router: matches command
- Command pipeline → Language-model providers: interprets fallback
- Command pipeline → Scenario runner: runs scenario
- Command pipeline → Action dispatcher: dispatches action
- Action dispatcher → Browser control: dispatches
- Action dispatcher → System actions: dispatches
- Action dispatcher → Connector actions: dispatches
- Connector actions → Vision and utility connectors: builds and calls
- Action dispatcher → Action plugins: loads extensions
- Listening controller → Speech output: speaks outcome
- Speech output → TTS backends: synthesizes speech
- TTS backends → Model and connector services: requests voice
- TTS backends → Optional TTS service: requests synthesis
- Listening controller → Conversation history: records turns
- Command pipeline → Assistant memory: uses memory actions
- Command pipeline → Configuration: reads settings
- Listening controller → Settings panel: updates activity
- User → Settings panel: controls assistant
- Settings panel → Listening controller: toggles pause
- System tray → Listening controller: controls assistant
- Browser control → Windows and apps: controls browser
- System actions → Windows and apps: controls system
- Language-model providers → Model and connector services: requests interpretation
- Vision and utility connectors → Model and connector services: requests services