# egjjr/speaky architecture

> Architecture diagram of the GitHub repository egjjr/speaky, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-09-30). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/egjjr/speaky
- Repository: https://github.com/egjjr/speaky

## Overview

Speaky is a macOS hold-to-talk voice companion: the user holds Control and Option, speaks while the microphone is active, and receives an action through the notch UI. The documented workflow runs from speech understanding and planning to Mac-native actions, with local confirmation for consequential operations. The graph preserves the documented app, web/link, notes, window-tiling, and search capabilities. Several implementation modules were not sampled, so their internal connections are omitted unless the README or source index supports them.

Main entry points: [`Sources/ForeApp/NotchModel.swift`](https://github.com/egjjr/speaky/blob/main/Sources/ForeApp/NotchModel.swift) (Turn coordinator), [`Sources/ForeCore/Executor.swift`](https://github.com/egjjr/speaky/blob/main/Sources/ForeCore/Executor.swift) (Action executor), [`Sources/ForeCore/Planner.swift`](https://github.com/egjjr/speaky/blob/main/Sources/ForeCore/Planner.swift) (Request planner), [`Sources/ForeApp/SpeechListener.swift`](https://github.com/egjjr/speaky/blob/main/Sources/ForeApp/SpeechListener.swift) (Speech capture)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_interaction["Voice interaction"]
  node_hold["Hold controls<br/>[HoldToTalk.swift]"]
  node_speech["Speech capture"]
  node_capture["Audio analyzer"]
  node_notch["Notch interface<br/>[NotchView.swift]"]
  node_turn["Turn coordinator<br/>[NotchModel.swift]"]
end

subgraph group_planning["Understanding and planning"]
  node_planner["Request planner<br/>[Planner.swift]"]
  node_stream["Streaming planner"]
  node_router["Screen-aware routing"]
  node_appnames["App catalog<br/>[AppCatalog.swift]"]
  node_live["Live computer<br/>[LiveComputer.swift]"]
end

subgraph group_actions["Mac actions"]
  node_executor["Action executor<br/>[Executor.swift]"]
  node_launch["App launching<br/>[SpokenApp.swift]"]
  node_link["Link following<br/>[LinkFollow.swift]"]
  node_notes["Notes task<br/>[NotesTask.swift]"]
  node_windows["Window tiling<br/>[WindowLayout.swift]"]
  node_desk["Search and desk<br/>[Desk.swift]"]
  node_accessibility["Accessibility actions"]
end

subgraph group_context["Context and safeguards"]
  node_context["Front-app context"]
  node_gate["Confirmation gate<br/>[ToolGate.swift]"]
  node_appcatalog[("Installed apps<br/>[AppCatalog.swift]")]
end

node_user(("Speaker"))
node_external(("Model services"))

node_user -->|"holds keys"| node_hold
node_hold -->|"starts capture"| node_speech
node_speech -->|"captures audio"| node_capture
node_speech -->|"provides transcript"| node_turn
node_turn -->|"updates status"| node_notch
node_turn -->|"plans request"| node_planner
node_turn -->|"streams decisions"| node_stream
node_planner -->|"checks screen context"| node_router
node_router -.->|"uses snapshot"| node_context
node_planner -.->|"resolves app names"| node_appnames
node_turn -->|"reads front context"| node_context
node_turn -->|"steps live plan"| node_live
node_turn -->|"executes plan"| node_executor
node_executor -->|"launches apps"| node_launch
node_executor -.->|"follows links"| node_link
node_executor -.->|"writes notes"| node_notes
node_executor -->|"tiles windows"| node_windows
node_turn -->|"runs desk tasks"| node_desk
node_executor -->|"uses accessibility"| node_accessibility
node_turn -->|"checks action risk"| node_gate
node_appcatalog -.->|"provides installed apps"| node_appnames
node_external -.->|"fills planning gaps"| node_planner
node_user -->|"confirms locally"| node_notch

click node_hold "https://github.com/egjjr/speaky/blob/main/Sources/ForeApp/HoldToTalk.swift"
click node_speech "https://github.com/egjjr/speaky/blob/main/Sources/ForeApp/SpeechListener.swift"
click node_capture "https://github.com/egjjr/speaky/blob/main/Sources/ForeApp/SpeechAnalyzerCapture.swift"
click node_notch "https://github.com/egjjr/speaky/blob/main/Sources/ForeApp/NotchView.swift"
click node_turn "https://github.com/egjjr/speaky/blob/main/Sources/ForeApp/NotchModel.swift"
click node_planner "https://github.com/egjjr/speaky/blob/main/Sources/ForeCore/Planner.swift"
click node_stream "https://github.com/egjjr/speaky/blob/main/Sources/ForeCore/StreamingPlanner.swift"
click node_router "https://github.com/egjjr/speaky/blob/main/Sources/ForeCore/ContextRouter.swift"
click node_appnames "https://github.com/egjjr/speaky/blob/main/Sources/ForeCore/AppCatalog.swift"
click node_context "https://github.com/egjjr/speaky/blob/main/Sources/ForeCore/SystemFrontContext.swift"
click node_live "https://github.com/egjjr/speaky/blob/main/Sources/ForeCore/LiveComputer.swift"
click node_executor "https://github.com/egjjr/speaky/blob/main/Sources/ForeCore/Executor.swift"
click node_launch "https://github.com/egjjr/speaky/blob/main/Sources/ForeCore/SpokenApp.swift"
click node_link "https://github.com/egjjr/speaky/blob/main/Sources/ForeCore/LinkFollow.swift"
click node_notes "https://github.com/egjjr/speaky/blob/main/Sources/ForeCore/NotesTask.swift"
click node_windows "https://github.com/egjjr/speaky/blob/main/Sources/ForeCore/WindowLayout.swift"
click node_desk "https://github.com/egjjr/speaky/blob/main/Sources/ForeCore/Desk.swift"
click node_accessibility "https://github.com/egjjr/speaky/blob/main/Sources/ForeCore/AccessibilityLinkFollower.swift"
click node_gate "https://github.com/egjjr/speaky/blob/main/Sources/ForeCore/ToolGate.swift"
click node_appcatalog "https://github.com/egjjr/speaky/blob/main/Sources/ForeCore/AppCatalog.swift"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_hold,node_speech,node_capture,node_notch,node_turn toneBlue
class node_planner,node_stream,node_router,node_appnames,node_live toneAmber
class node_executor,node_launch,node_link,node_notes,node_windows,node_desk,node_accessibility toneMint
class node_context,node_gate,node_appcatalog toneRose
class node_user,node_external toneIndigo
```

## Components

### Voice interaction

- [Hold controls](https://github.com/egjjr/speaky/blob/main/Sources/ForeApp/HoldToTalk.swift): `Sources/ForeApp/HoldToTalk.swift`
- [Speech capture](https://github.com/egjjr/speaky/blob/main/Sources/ForeApp/SpeechListener.swift): `Sources/ForeApp/SpeechListener.swift`
- [Audio analyzer](https://github.com/egjjr/speaky/blob/main/Sources/ForeApp/SpeechAnalyzerCapture.swift): `Sources/ForeApp/SpeechAnalyzerCapture.swift`
- [Notch interface](https://github.com/egjjr/speaky/blob/main/Sources/ForeApp/NotchView.swift): `Sources/ForeApp/NotchView.swift`
- [Turn coordinator](https://github.com/egjjr/speaky/blob/main/Sources/ForeApp/NotchModel.swift): `Sources/ForeApp/NotchModel.swift`

### Understanding and planning

- [Request planner](https://github.com/egjjr/speaky/blob/main/Sources/ForeCore/Planner.swift): `Sources/ForeCore/Planner.swift`
- [Streaming planner](https://github.com/egjjr/speaky/blob/main/Sources/ForeCore/StreamingPlanner.swift): `Sources/ForeCore/StreamingPlanner.swift`
- [Screen-aware routing](https://github.com/egjjr/speaky/blob/main/Sources/ForeCore/ContextRouter.swift): `Sources/ForeCore/ContextRouter.swift`
- [App catalog](https://github.com/egjjr/speaky/blob/main/Sources/ForeCore/AppCatalog.swift): `Sources/ForeCore/AppCatalog.swift`
- [Live computer](https://github.com/egjjr/speaky/blob/main/Sources/ForeCore/LiveComputer.swift): `Sources/ForeCore/LiveComputer.swift`

### Mac actions

- [Action executor](https://github.com/egjjr/speaky/blob/main/Sources/ForeCore/Executor.swift): `Sources/ForeCore/Executor.swift`
- [App launching](https://github.com/egjjr/speaky/blob/main/Sources/ForeCore/SpokenApp.swift): `Sources/ForeCore/SpokenApp.swift`
- [Link following](https://github.com/egjjr/speaky/blob/main/Sources/ForeCore/LinkFollow.swift): `Sources/ForeCore/LinkFollow.swift`
- [Notes task](https://github.com/egjjr/speaky/blob/main/Sources/ForeCore/NotesTask.swift): `Sources/ForeCore/NotesTask.swift`
- [Window tiling](https://github.com/egjjr/speaky/blob/main/Sources/ForeCore/WindowLayout.swift): `Sources/ForeCore/WindowLayout.swift`
- [Search and desk](https://github.com/egjjr/speaky/blob/main/Sources/ForeCore/Desk.swift): `Sources/ForeCore/Desk.swift`
- [Accessibility actions](https://github.com/egjjr/speaky/blob/main/Sources/ForeCore/AccessibilityLinkFollower.swift): `Sources/ForeCore/AccessibilityLinkFollower.swift`

### Context and safeguards

- [Front-app context](https://github.com/egjjr/speaky/blob/main/Sources/ForeCore/SystemFrontContext.swift): `Sources/ForeCore/SystemFrontContext.swift`
- [Confirmation gate](https://github.com/egjjr/speaky/blob/main/Sources/ForeCore/ToolGate.swift): `Sources/ForeCore/ToolGate.swift`
- [Installed apps](https://github.com/egjjr/speaky/blob/main/Sources/ForeCore/AppCatalog.swift): `Sources/ForeCore/AppCatalog.swift`

### Other components

- **Speaker**
- **Model services**

## Connections

- Speaker → Hold controls: holds keys
- Hold controls → Speech capture: starts capture (evidence: `Sources/ForeApp/HoldToTalk.swift`)
- Speech capture → Audio analyzer: captures audio (evidence: `Sources/ForeApp/SpeechListener.swift`)
- Speech capture → Turn coordinator: provides transcript (evidence: `Sources/ForeApp/NotchModel.swift`)
- Turn coordinator → Notch interface: updates status (evidence: `Sources/ForeApp/NotchView.swift`)
- Turn coordinator → Request planner: plans request (evidence: `Sources/ForeApp/NotchModel.swift`)
- Turn coordinator → Streaming planner: streams decisions (evidence: `Sources/ForeApp/NotchModel.swift`)
- Request planner → Screen-aware routing: checks screen context (evidence: `Sources/ForeCore/Planner.swift`)
- Screen-aware routing → Front-app context: uses snapshot
- Request planner → App catalog: resolves app names (evidence: `Sources/ForeCore/Planner.swift`)
- Turn coordinator → Front-app context: reads front context (evidence: `Sources/ForeApp/NotchModel.swift`)
- Turn coordinator → Live computer: steps live plan (evidence: `Sources/ForeApp/NotchModel.swift`)
- Turn coordinator → Action executor: executes plan (evidence: `Sources/ForeApp/NotchModel.swift`)
- Action executor → App launching: launches apps (evidence: `Sources/ForeCore/Executor.swift`)
- Action executor → Link following: follows links
- Action executor → Notes task: writes notes
- Action executor → Window tiling: tiles windows (evidence: `Sources/ForeCore/Executor.swift`)
- Turn coordinator → Search and desk: runs desk tasks (evidence: `Sources/ForeApp/NotchModel.swift`)
- Action executor → Accessibility actions: uses accessibility (evidence: `Sources/ForeCore/Executor.swift`)
- Turn coordinator → Confirmation gate: checks action risk (evidence: `Sources/ForeApp/NotchModel.swift`)
- Installed apps → App catalog: provides installed apps (evidence: `Sources/ForeCore/AppCatalog.swift`)
- Model services → Request planner: fills planning gaps
- Speaker → Notch interface: confirms locally