# zisshh/computer-use architecture

> Architecture diagram of the GitHub repository zisshh/computer-use, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-10-03). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/zisshh/computer-use
- Repository: https://github.com/zisshh/computer-use

## Overview

Alfred is a macOS voice-control assistant: it captures speech, transcribes it locally, interprets commands, and executes actions on the Mac or in supported apps. The main path runs through the session handler, which can use fast local/page-aware handling or ask Jev to select a typed action plan before execution. Supporting branches cover chat and YouTube-specific commands, audio gating and echo cancellation, screen context, and spoken/visual feedback. The README documents capabilities beyond the sampled excerpts; edges for those features cite it rather than implying unsampled implementation details were inspected.

Main entry points: [`alfred_computer_use/main.py`](https://github.com/zisshh/computer-use/blob/main/alfred_computer_use/main.py) (Session handler), [`alfred_computer_use/audio.py`](https://github.com/zisshh/computer-use/blob/main/alfred_computer_use/audio.py) (Audio processing), [`alfred_computer_use/brain.py`](https://github.com/zisshh/computer-use/blob/main/alfred_computer_use/brain.py) (Intent planner), [`alfred_computer_use/actions.py`](https://github.com/zisshh/computer-use/blob/main/alfred_computer_use/actions.py) (macOS actions)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_voice["Voice input"]
  node_session["Session handler<br/>[main.py]"]
  node_capture["Audio capture<br/>[recorder.py]"]
  node_audio["Audio processing<br/>[audio.py]"]
  node_aec["Echo cancellation<br/>[aec.py]"]
  node_vad["Speech gate<br/>[vad.py]"]
  node_stt["Speech recognition<br/>[stt.py]"]
  node_apple_stt["Apple recognition<br/>[apple_stt.py]"]
  node_streaming["Streaming listener<br/>[streaming.py]"]
end

subgraph group_intent["Command understanding"]
  node_local["Local command handling<br/>[local.py]"]
  node_brain["Intent planner<br/>[brain.py]"]
  node_routing["Command routing<br/>[routing.py]"]
  node_catalog[("Entity catalog<br/>[catalog.py]")]
  node_chat_intent["Chat intent<br/>[chat_intent.py]"]
end

subgraph group_execution["Action execution"]
  node_executor["Action executor<br/>[main.py]"]
  node_actions["macOS actions<br/>[actions.py]"]
  node_osa["Apple Events<br/>[osa.py]"]
end

subgraph group_experience["Context and feedback"]
  node_screen["Screen context<br/>[context.py]"]
  node_focus["App focus<br/>[focus.py]"]
  node_overlay["Status overlay<br/>[overlay.py]"]
  node_speech["Spoken replies<br/>[tts.py]"]
  node_config["Runtime configuration<br/>[config.py]"]
end

subgraph group_integrations["App integrations"]
  node_chat["Messages and WhatsApp<br/>[chat.py]"]
  node_discord["Discord controls<br/>[discord.py]"]
  node_youtube["YouTube controls<br/>[youtube.py]"]
  node_page["Page interaction<br/>[page.py]"]
end

node_user(("User"))
node_jev{{"Jev decision service"}}
node_spotify{{"Spotify service"}}

node_user -->|"speaks to"| node_session
node_session -->|"captures audio"| node_capture
node_capture -.->|"provides audio"| node_audio
node_audio -->|"cancels echo"| node_aec
node_audio -.->|"gates speech"| node_vad
node_session -->|"starts listener"| node_streaming
node_streaming -->|"transcribes audio"| node_stt
node_stt -->|"selects engine"| node_apple_stt
node_session -->|"checks locally"| node_local
node_session -->|"plans command"| node_brain
node_brain -->|"requests selection"| node_jev
node_brain -->|"selects entities"| node_catalog
node_session -->|"resolves target"| node_routing
node_session -->|"dispatches plan"| node_executor
node_executor -->|"executes actions"| node_actions
node_actions -->|"sends Apple events"| node_osa
node_executor -->|"reads context"| node_screen
node_executor -->|"checks focus"| node_focus
node_executor -->|"handles chat"| node_chat
node_executor -->|"parses chat request"| node_chat_intent
node_executor -->|"controls Discord"| node_discord
node_executor -->|"handles YouTube"| node_youtube
node_executor -->|"acts on page"| node_page
node_actions -.->|"plays selected media"| node_spotify
node_session -->|"updates status"| node_overlay
node_session -->|"speaks reply"| node_speech
node_session -->|"loads settings"| node_config

click node_session "https://github.com/zisshh/computer-use/blob/main/alfred_computer_use/main.py"
click node_capture "https://github.com/zisshh/computer-use/blob/main/alfred_computer_use/recorder.py"
click node_audio "https://github.com/zisshh/computer-use/blob/main/alfred_computer_use/audio.py"
click node_aec "https://github.com/zisshh/computer-use/blob/main/alfred_computer_use/aec.py"
click node_vad "https://github.com/zisshh/computer-use/blob/main/alfred_computer_use/vad.py"
click node_stt "https://github.com/zisshh/computer-use/blob/main/alfred_computer_use/stt.py"
click node_apple_stt "https://github.com/zisshh/computer-use/blob/main/alfred_computer_use/apple_stt.py"
click node_streaming "https://github.com/zisshh/computer-use/blob/main/alfred_computer_use/streaming.py"
click node_local "https://github.com/zisshh/computer-use/blob/main/alfred_computer_use/local.py"
click node_brain "https://github.com/zisshh/computer-use/blob/main/alfred_computer_use/brain.py"
click node_routing "https://github.com/zisshh/computer-use/blob/main/alfred_computer_use/routing.py"
click node_catalog "https://github.com/zisshh/computer-use/blob/main/alfred_computer_use/catalog.py"
click node_chat_intent "https://github.com/zisshh/computer-use/blob/main/alfred_computer_use/chat_intent.py"
click node_executor "https://github.com/zisshh/computer-use/blob/main/alfred_computer_use/main.py"
click node_actions "https://github.com/zisshh/computer-use/blob/main/alfred_computer_use/actions.py"
click node_osa "https://github.com/zisshh/computer-use/blob/main/alfred_computer_use/osa.py"
click node_screen "https://github.com/zisshh/computer-use/blob/main/alfred_computer_use/context.py"
click node_focus "https://github.com/zisshh/computer-use/blob/main/alfred_computer_use/focus.py"
click node_overlay "https://github.com/zisshh/computer-use/blob/main/alfred_computer_use/overlay.py"
click node_speech "https://github.com/zisshh/computer-use/blob/main/alfred_computer_use/tts.py"
click node_chat "https://github.com/zisshh/computer-use/blob/main/alfred_computer_use/chat.py"
click node_discord "https://github.com/zisshh/computer-use/blob/main/alfred_computer_use/discord.py"
click node_youtube "https://github.com/zisshh/computer-use/blob/main/alfred_computer_use/youtube.py"
click node_page "https://github.com/zisshh/computer-use/blob/main/alfred_computer_use/page.py"
click node_config "https://github.com/zisshh/computer-use/blob/main/alfred_computer_use/config.py"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_session,node_capture,node_audio,node_aec,node_vad,node_stt,node_apple_stt,node_streaming,node_user toneBlue
class node_local,node_brain,node_routing,node_catalog,node_chat_intent toneAmber
class node_executor,node_actions,node_osa toneMint
class node_screen,node_focus,node_overlay,node_speech,node_config toneRose
class node_chat,node_discord,node_youtube,node_page,node_jev,node_spotify toneIndigo
```

## Components

### Voice input

- [Session handler](https://github.com/zisshh/computer-use/blob/main/alfred_computer_use/main.py): `alfred_computer_use/main.py`
- [Audio capture](https://github.com/zisshh/computer-use/blob/main/alfred_computer_use/recorder.py): `alfred_computer_use/recorder.py`
- [Audio processing](https://github.com/zisshh/computer-use/blob/main/alfred_computer_use/audio.py): `alfred_computer_use/audio.py`
- [Echo cancellation](https://github.com/zisshh/computer-use/blob/main/alfred_computer_use/aec.py): `alfred_computer_use/aec.py`
- [Speech gate](https://github.com/zisshh/computer-use/blob/main/alfred_computer_use/vad.py): `alfred_computer_use/vad.py`
- [Speech recognition](https://github.com/zisshh/computer-use/blob/main/alfred_computer_use/stt.py): `alfred_computer_use/stt.py`
- [Apple recognition](https://github.com/zisshh/computer-use/blob/main/alfred_computer_use/apple_stt.py): `alfred_computer_use/apple_stt.py`
- [Streaming listener](https://github.com/zisshh/computer-use/blob/main/alfred_computer_use/streaming.py): `alfred_computer_use/streaming.py`

### Command understanding

- [Local command handling](https://github.com/zisshh/computer-use/blob/main/alfred_computer_use/local.py): `alfred_computer_use/local.py`
- [Intent planner](https://github.com/zisshh/computer-use/blob/main/alfred_computer_use/brain.py): `alfred_computer_use/brain.py`
- [Command routing](https://github.com/zisshh/computer-use/blob/main/alfred_computer_use/routing.py): `alfred_computer_use/routing.py`
- [Entity catalog](https://github.com/zisshh/computer-use/blob/main/alfred_computer_use/catalog.py): `alfred_computer_use/catalog.py`
- [Chat intent](https://github.com/zisshh/computer-use/blob/main/alfred_computer_use/chat_intent.py): `alfred_computer_use/chat_intent.py`

### Action execution

- [Action executor](https://github.com/zisshh/computer-use/blob/main/alfred_computer_use/main.py): `alfred_computer_use/main.py`
- [macOS actions](https://github.com/zisshh/computer-use/blob/main/alfred_computer_use/actions.py): `alfred_computer_use/actions.py`
- [Apple Events](https://github.com/zisshh/computer-use/blob/main/alfred_computer_use/osa.py): `alfred_computer_use/osa.py`

### Context and feedback

- [Screen context](https://github.com/zisshh/computer-use/blob/main/alfred_computer_use/context.py): `alfred_computer_use/context.py`
- [App focus](https://github.com/zisshh/computer-use/blob/main/alfred_computer_use/focus.py): `alfred_computer_use/focus.py`
- [Status overlay](https://github.com/zisshh/computer-use/blob/main/alfred_computer_use/overlay.py): `alfred_computer_use/overlay.py`
- [Spoken replies](https://github.com/zisshh/computer-use/blob/main/alfred_computer_use/tts.py): `alfred_computer_use/tts.py`
- [Runtime configuration](https://github.com/zisshh/computer-use/blob/main/alfred_computer_use/config.py): `alfred_computer_use/config.py`

### App integrations

- [Messages and WhatsApp](https://github.com/zisshh/computer-use/blob/main/alfred_computer_use/chat.py): `alfred_computer_use/chat.py`
- [Discord controls](https://github.com/zisshh/computer-use/blob/main/alfred_computer_use/discord.py): `alfred_computer_use/discord.py`
- [YouTube controls](https://github.com/zisshh/computer-use/blob/main/alfred_computer_use/youtube.py): `alfred_computer_use/youtube.py`
- [Page interaction](https://github.com/zisshh/computer-use/blob/main/alfred_computer_use/page.py): `alfred_computer_use/page.py`

### Other components

- **User**
- **Jev decision service**
- **Spotify service**

## Connections

- User → Session handler: speaks to (evidence: `README.md`)
- Session handler → Audio capture: captures audio (evidence: `alfred_computer_use/main.py`)
- Audio capture → Audio processing: provides audio
- Audio processing → Echo cancellation: cancels echo (evidence: `alfred_computer_use/audio.py`)
- Audio processing → Speech gate: gates speech
- Session handler → Streaming listener: starts listener (evidence: `alfred_computer_use/main.py`)
- Streaming listener → Speech recognition: transcribes audio (evidence: `alfred_computer_use/streaming.py`)
- Speech recognition → Apple recognition: selects engine (evidence: `alfred_computer_use/stt.py`)
- Session handler → Local command handling: checks locally (evidence: `alfred_computer_use/main.py`)
- Session handler → Intent planner: plans command (evidence: `alfred_computer_use/main.py`)
- Intent planner → Jev decision service: requests selection (evidence: `README.md`)
- Intent planner → Entity catalog: selects entities (evidence: `alfred_computer_use/brain.py`)
- Session handler → Command routing: resolves target (evidence: `alfred_computer_use/main.py`)
- Session handler → Action executor: dispatches plan (evidence: `alfred_computer_use/main.py`)
- Action executor → macOS actions: executes actions (evidence: `alfred_computer_use/main.py`)
- macOS actions → Apple Events: sends Apple events (evidence: `alfred_computer_use/actions.py`)
- Action executor → Screen context: reads context (evidence: `alfred_computer_use/main.py`)
- Action executor → App focus: checks focus (evidence: `alfred_computer_use/main.py`)
- Action executor → Messages and WhatsApp: handles chat (evidence: `alfred_computer_use/main.py`)
- Action executor → Chat intent: parses chat request
- Action executor → Discord controls: controls Discord (evidence: `alfred_computer_use/main.py`)
- Action executor → YouTube controls: handles YouTube
- Action executor → Page interaction: acts on page
- macOS actions → Spotify service: plays selected media (evidence: `README.md`)
- Session handler → Status overlay: updates status (evidence: `alfred_computer_use/main.py`)
- Session handler → Spoken replies: speaks reply (evidence: `README.md`)
- Session handler → Runtime configuration: loads settings (evidence: `alfred_computer_use/main.py`)
