# farzaa/clicky architecture

> Architecture diagram of the GitHub repository farzaa/clicky, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-05-01). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/farzaa/clicky
- Repository: https://github.com/farzaa/clicky

## Overview

1. **Overall shape**
   - This repo is a macOS desktop app plus a small Cloudflare Worker backend. The app is the product; the Worker is a thin proxy layer that keeps third-party API keys out of the client binary.

2. **Primary runtime boundary**
   - `leanring-buddy/` is the Swift app target.
   - `worker/` is a TypeScript Cloudflare Worker deployed separately.
   - The app never talks directly to Anthropic, AssemblyAI, or ElevenLabs; it calls the Worker, which forwards requests to those vendors.

3. **macOS app shell**
   - The app is a menu bar utility, not a normal docked app.
   - `leanring_buddyApp.swift`, `MenuBarPanelManager.swift`, and `CompanionPanelView.swift` coordinate the app lifecycle and the menu-bar UI.
   - `OverlayWindow.swift` provides the transparent/full-screen overlay used to draw attention to UI elements near the cursor.

4. **Central orchestration**
   - `CompanionManager.swift` is the main state machine and coordination layer.
   - It ties together transcription, Claude chat, screen capture, TTS playback, and overlay behavior.
   - Most user-facing flows converge here, which makes it the best single point for understanding request/response sequencing.

5. **Voice input pipeline**
   - Push-to-talk is handled by `GlobalPushToTalkShortcutMonitor.swift`, `BuddyDictationManager.swift`, and `BuddyAudioConversionSupport.swift`.
   - Audio is captured from the microphone, converted as needed, and streamed to transcription services.
   - The app supports more than one transcription provider: `AssemblyAIStreamingTranscriptionProvider.swift`, `OpenAIAudioTranscriptionProvider.swift`, and `AppleSpeechTranscriptionProvider.swift`, all conforming to `BuddyTranscriptionProvider.swift`.

6. **Transcription backend flow**
   - For real-time dictation, the app requests a short-lived AssemblyAI token from the Worker and then streams audio to AssemblyAI.
   - This is a good example of the repo’s trust boundary: sensitive credentials stay server-side, while the client only receives scoped runtime tokens or proxied responses.

7. **Claude + response generation**
   - `ClaudeAPI.swift` sends the transcript plus any captured screen context to Claude through the Worker’s `/chat` route.
   - Claude responses can include structured point-to-screen instructions, which the app interprets to move or highlight attention on the display.
   - `ElementLocationDetector.swift` and `CompanionScreenCaptureUtility.swift` help translate screen information into actionable coordinates.

8. **Speech output**
   - `ElevenLabsTTSClient.swift` turns Claude’s response text into audio via the Worker’s `/tts` route.
   - The app includes local sound assets like `enter.mp3`, `eshop.mp3`, and `ff.mp3` for UI/audio cues in addition to streamed TTS.

9. **Screen interaction and pointing**
   - `OverlayWindow.swift`, `CompanionResponseOverlay.swift`, and `WindowPositionManager.swift` manage visual positioning relative to the cursor and display geometry.
   - The overlay subsystem is separate from the main panel so the app can present guidance without taking over the whole desktop.
   - Multi-monitor awareness is implied by the screen-location handling in the response format.

10. **Analytics, configuration, and distribution**
   - `ClickyAnalytics.swift` suggests lightweight product telemetry.
   - `AppBundleConfiguration.swift` and `Info.plist` define app-level packaging and runtime settings.
   - `appcast.xml` and `scripts/release.sh` indicate an update/release workflow, likely for app distribution outside the App Store.

11. **Worker structure**
   - `worker/src/index.ts` is the Worker entrypoint; `wrangler.toml` defines deployment configuration.
   - The Worker exposes three main proxy responsibilities: Claude chat, ElevenLabs TTS, and AssemblyAI transcription-token issuance.
   - Its job is deliberately small: route, authenticate with secret keys, and return vendor responses to the macOS client.

12. **Operational dependencies**
   - The system depends on macOS permissions for microphone, accessibility, and screen recording / screen content access.
   - At runtime the app relies on ScreenCaptureKit-style screen access, a global keyboard shortcut, and network connectivity to the Worker and downstream AI services.

13. **What an architecture diagram should show**
   - A left-to-right flow works well: user input on the Mac app, optional screen capture, transcription, Claude reasoning, TTS output, and overlay feedback.
   - Place the Cloudflare Worker as the security boundary in the middle, with arrows from the app to the Worker and from the Worker to Anthropic, AssemblyAI, and ElevenLabs.

Main entry points: [`leanring-buddy/CompanionManager.swift`](https://github.com/farzaa/clicky/blob/main/leanring-buddy/CompanionManager.swift) (Companion manager), [`worker/src/index.ts`](https://github.com/farzaa/clicky/blob/main/worker/src/index.ts) (Worker entry), [`leanring-buddy/BuddyDictationManager.swift`](https://github.com/farzaa/clicky/blob/main/leanring-buddy/BuddyDictationManager.swift) (Dictation flow), [`leanring-buddy/BuddyTranscriptionProvider.swift`](https://github.com/farzaa/clicky/blob/main/leanring-buddy/BuddyTranscriptionProvider.swift) (Transcription API)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_app["macOS App"]
  node_app_main["App entry<br/>Swift app"]
  node_app_menu["Menu bar<br/>UI shell"]
  node_app_panel["Companion panel<br/>SwiftUI view"]
  node_app_manager{{"Companion manager<br/>orchestrator"}}
  node_app_overlay["Overlay window<br/>overlay UI"]
  node_app_response_overlay["Response overlay<br/>overlay UI"]
  node_app_position["Window position<br/>layout"]
  node_app_dictation{{"Dictation flow<br/>audio pipeline"}}
  node_app_shortcut["Push-to-talk<br/>input monitor"]
  node_app_audio["Audio convert<br/>audio utils"]
  node_app_transcription["Transcription API<br/>provider protocol"]
  node_app_assembly["AssemblyAI stream<br/>transcription provider"]
  node_app_openai_transcribe["OpenAI speech<br/>transcription provider"]
  node_app_apple_speech["Apple speech<br/>transcription provider"]
  node_app_claude["Claude client<br/>chat client<br/>[ClaudeAPI.swift]"]
  node_app_tts["TTS client<br/>speech client"]
  node_app_screen["Screen capture"]
  node_app_element["Element lookup<br/>screen analysis"]
  node_app_analytics["Analytics<br/>telemetry"]
end

subgraph group_worker["Cloudflare Worker"]
  node_worker_entry{{"Worker entry<br/>[index.ts]"}}
  node_worker_cfg["Worker config<br/>deployment config<br/>[wrangler.toml]"]
end

subgraph group_external["External Services"]
  node_vendor_anthropic(("Anthropic<br/>LLM API"))
  node_vendor_assembly(("AssemblyAI<br/>speech API"))
  node_vendor_eleven(("ElevenLabs<br/>TTS API"))
  node_sys_mac(("macOS permissions<br/>OS boundary"))
end

node_app_main -->|"boots UI"| node_app_menu
node_app_menu -->|"shows"| node_app_panel
node_app_menu -->|"owns"| node_app_manager
node_app_manager -->|"listens"| node_app_shortcut
node_app_manager -->|"starts capture"| node_app_dictation
node_app_dictation -->|"normalizes"| node_app_audio
node_app_dictation -->|"uses"| node_app_transcription
node_app_transcription -->|"implements"| node_app_assembly
node_app_transcription -->|"implements"| node_app_openai_transcribe
node_app_transcription -->|"implements"| node_app_apple_speech
node_app_manager -->|"captures context"| node_app_screen
node_app_screen -->|"analyzes"| node_app_element
node_app_manager -->|"sends prompt"| node_app_claude
node_app_manager -->|"speaks reply"| node_app_tts
node_app_manager -->|"highlights"| node_app_overlay
node_app_manager -->|"renders response"| node_app_response_overlay
node_app_manager -->|"positions"| node_app_position
node_app_manager -->|"logs"| node_app_analytics
node_app_claude -->|"proxied via"| node_worker_entry
node_app_tts -->|"proxied via"| node_worker_entry
node_app_dictation -->|"token request"| node_worker_entry
node_worker_entry -->|"forwards"| node_vendor_anthropic
node_worker_entry -->|"issues token"| node_vendor_assembly
node_worker_entry -->|"forwards"| node_vendor_eleven
node_app_manager -->|"depends on"| node_sys_mac

click node_app_main "https://github.com/farzaa/clicky/blob/main/leanring-buddy/leanring_buddyApp.swift"
click node_app_menu "https://github.com/farzaa/clicky/blob/main/leanring-buddy/MenuBarPanelManager.swift"
click node_app_panel "https://github.com/farzaa/clicky/blob/main/leanring-buddy/CompanionPanelView.swift"
click node_app_manager "https://github.com/farzaa/clicky/blob/main/leanring-buddy/CompanionManager.swift"
click node_app_overlay "https://github.com/farzaa/clicky/blob/main/leanring-buddy/OverlayWindow.swift"
click node_app_response_overlay "https://github.com/farzaa/clicky/blob/main/leanring-buddy/CompanionResponseOverlay.swift"
click node_app_position "https://github.com/farzaa/clicky/blob/main/leanring-buddy/WindowPositionManager.swift"
click node_app_dictation "https://github.com/farzaa/clicky/blob/main/leanring-buddy/BuddyDictationManager.swift"
click node_app_shortcut "https://github.com/farzaa/clicky/blob/main/leanring-buddy/GlobalPushToTalkShortcutMonitor.swift"
click node_app_audio "https://github.com/farzaa/clicky/blob/main/leanring-buddy/BuddyAudioConversionSupport.swift"
click node_app_transcription "https://github.com/farzaa/clicky/blob/main/leanring-buddy/BuddyTranscriptionProvider.swift"
click node_app_assembly "https://github.com/farzaa/clicky/blob/main/leanring-buddy/AssemblyAIStreamingTranscriptionProvider.swift"
click node_app_openai_transcribe "https://github.com/farzaa/clicky/blob/main/leanring-buddy/OpenAIAudioTranscriptionProvider.swift"
click node_app_apple_speech "https://github.com/farzaa/clicky/blob/main/leanring-buddy/AppleSpeechTranscriptionProvider.swift"
click node_app_claude "https://github.com/farzaa/clicky/blob/main/leanring-buddy/ClaudeAPI.swift"
click node_app_tts "https://github.com/farzaa/clicky/blob/main/leanring-buddy/ElevenLabsTTSClient.swift"
click node_app_screen "https://github.com/farzaa/clicky/blob/main/leanring-buddy/CompanionScreenCaptureUtility.swift"
click node_app_element "https://github.com/farzaa/clicky/blob/main/leanring-buddy/ElementLocationDetector.swift"
click node_app_analytics "https://github.com/farzaa/clicky/blob/main/leanring-buddy/ClickyAnalytics.swift"
click node_worker_entry "https://github.com/farzaa/clicky/blob/main/worker/src/index.ts"
click node_worker_cfg "https://github.com/farzaa/clicky/blob/main/worker/wrangler.toml"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_app_main,node_app_menu,node_app_panel,node_app_manager,node_app_overlay,node_app_response_overlay,node_app_position,node_app_dictation,node_app_shortcut,node_app_audio,node_app_transcription,node_app_assembly,node_app_openai_transcribe,node_app_apple_speech,node_app_claude,node_app_tts,node_app_screen,node_app_element,node_app_analytics toneBlue
class node_worker_entry,node_worker_cfg toneAmber
class node_vendor_anthropic,node_vendor_assembly,node_vendor_eleven,node_sys_mac toneMint
```

## Components

### macOS App

Swift menu-bar desktop client

- [App entry](https://github.com/farzaa/clicky/blob/main/leanring-buddy/leanring_buddyApp.swift): Swift app · Launches the menu-bar app lifecycle · `leanring-buddy/leanring_buddyApp.swift`
- [Menu bar](https://github.com/farzaa/clicky/blob/main/leanring-buddy/MenuBarPanelManager.swift): UI shell · Menu-bar panel and companion UI shell · `leanring-buddy/MenuBarPanelManager.swift`
- [Companion panel](https://github.com/farzaa/clicky/blob/main/leanring-buddy/CompanionPanelView.swift): SwiftUI view · Primary visible companion panel · `leanring-buddy/CompanionPanelView.swift`
- [Companion manager](https://github.com/farzaa/clicky/blob/main/leanring-buddy/CompanionManager.swift): orchestrator · Main state machine coordinating the user flow · `leanring-buddy/CompanionManager.swift`
- [Overlay window](https://github.com/farzaa/clicky/blob/main/leanring-buddy/OverlayWindow.swift): overlay UI · Transparent attention overlay near the cursor · `leanring-buddy/OverlayWindow.swift`
- [Response overlay](https://github.com/farzaa/clicky/blob/main/leanring-buddy/CompanionResponseOverlay.swift): overlay UI · Renders point-to-screen guidance and highlights · `leanring-buddy/CompanionResponseOverlay.swift`
- [Window position](https://github.com/farzaa/clicky/blob/main/leanring-buddy/WindowPositionManager.swift): layout · Positions panels and overlays across displays · `leanring-buddy/WindowPositionManager.swift`
- [Dictation flow](https://github.com/farzaa/clicky/blob/main/leanring-buddy/BuddyDictationManager.swift): audio pipeline · Captures and prepares microphone audio for transcription · `leanring-buddy/BuddyDictationManager.swift`
- [Push-to-talk](https://github.com/farzaa/clicky/blob/main/leanring-buddy/GlobalPushToTalkShortcutMonitor.swift): input monitor · Global keyboard shortcut for voice capture · `leanring-buddy/GlobalPushToTalkShortcutMonitor.swift`
- [Audio convert](https://github.com/farzaa/clicky/blob/main/leanring-buddy/BuddyAudioConversionSupport.swift): audio utils · Audio conversion and format support · `leanring-buddy/BuddyAudioConversionSupport.swift`
- [Transcription API](https://github.com/farzaa/clicky/blob/main/leanring-buddy/BuddyTranscriptionProvider.swift): provider protocol · Common interface for transcription providers · `leanring-buddy/BuddyTranscriptionProvider.swift`
- [AssemblyAI stream](https://github.com/farzaa/clicky/blob/main/leanring-buddy/AssemblyAIStreamingTranscriptionProvider.swift): transcription provider · Real-time streaming transcription provider · `leanring-buddy/AssemblyAIStreamingTranscriptionProvider.swift`
- [OpenAI speech](https://github.com/farzaa/clicky/blob/main/leanring-buddy/OpenAIAudioTranscriptionProvider.swift): transcription provider · Alternative audio transcription provider · `leanring-buddy/OpenAIAudioTranscriptionProvider.swift`
- [Apple speech](https://github.com/farzaa/clicky/blob/main/leanring-buddy/AppleSpeechTranscriptionProvider.swift): transcription provider · Local Apple speech transcription provider · `leanring-buddy/AppleSpeechTranscriptionProvider.swift`
- [Claude client](https://github.com/farzaa/clicky/blob/main/leanring-buddy/ClaudeAPI.swift): chat client · Sends transcript and screen context for reasoning · `leanring-buddy/ClaudeAPI.swift`
- [TTS client](https://github.com/farzaa/clicky/blob/main/leanring-buddy/ElevenLabsTTSClient.swift): speech client · Turns responses into audio playback · `leanring-buddy/ElevenLabsTTSClient.swift`
- [Screen capture](https://github.com/farzaa/clicky/blob/main/leanring-buddy/CompanionScreenCaptureUtility.swift): screen capture · Collects screen context for prompts and pointing · `leanring-buddy/CompanionScreenCaptureUtility.swift`
- [Element lookup](https://github.com/farzaa/clicky/blob/main/leanring-buddy/ElementLocationDetector.swift): screen analysis · Maps screen content to actionable coordinates · `leanring-buddy/ElementLocationDetector.swift`
- [Analytics](https://github.com/farzaa/clicky/blob/main/leanring-buddy/ClickyAnalytics.swift): telemetry · Lightweight product analytics · `leanring-buddy/ClickyAnalytics.swift`

### Cloudflare Worker

Thin proxy and token boundary

- [Worker entry](https://github.com/farzaa/clicky/blob/main/worker/src/index.ts): worker entrypoint · Cloudflare Worker request router · `worker/src/index.ts`
- [Worker config](https://github.com/farzaa/clicky/blob/main/worker/wrangler.toml): deployment config · Deployment and bindings configuration · `worker/wrangler.toml`

### External Services

Vendor APIs and system inputs

- **Anthropic**: LLM API · Claude chat backend
- **AssemblyAI**: speech API · Streaming transcription backend
- **ElevenLabs**: TTS API · Text-to-speech backend
- **macOS permissions**: OS boundary · Microphone, accessibility, and screen recording access

## Connections

- App entry → Menu bar: boots UI · Launches the menu-bar shell
- Menu bar → Companion panel: shows · Presents the companion panel
- Menu bar → Companion manager: owns · Creates and coordinates app state
- Companion manager → Push-to-talk: listens · Waits for push-to-talk input
- Companion manager → Dictation flow: starts capture · Begins microphone capture and transcription flow
- Dictation flow → Audio convert: normalizes · Prepares audio for providers
- Dictation flow → Transcription API: uses · Routes audio through the provider interface
- Transcription API → AssemblyAI stream: implements · Streaming AssemblyAI provider
- Transcription API → OpenAI speech: implements · OpenAI transcription provider
- Transcription API → Apple speech: implements · Apple speech provider
- Companion manager → Screen capture: captures context · Collects screen context for reasoning
- Screen capture → Element lookup: analyzes · Turns capture into screen targets
- Companion manager → Claude client: sends prompt · Combines transcript and screen context for Claude
- Companion manager → TTS client: speaks reply · Converts response text to audio
- Companion manager → Overlay window: highlights · Shows guidance overlay near cursor
- Companion manager → Response overlay: renders response · Displays point-to-screen instructions
- Companion manager → Window position: positions · Keeps windows aligned across displays
- Companion manager → Analytics: logs · Records lightweight telemetry
- Claude client → Worker entry: proxied via · Sends chat requests through the worker
- TTS client → Worker entry: proxied via · Requests TTS through the worker
- Dictation flow → Worker entry: token request · Fetches short-lived transcription token
- Worker entry → Anthropic: forwards · Proxies chat completion calls
- Worker entry → AssemblyAI: issues token · Returns scoped AssemblyAI token
- Worker entry → ElevenLabs: forwards · Proxies text-to-speech calls
- Companion manager → macOS permissions: depends on · Requires OS-level capture and shortcut permissions
