# themanojdesai/clicky architecture

> Architecture diagram of the GitHub repository themanojdesai/clicky, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-05-23). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/themanojdesai/clicky
- Repository: https://github.com/themanojdesai/clicky

## Overview

1. **Overall shape**
   Clicky is a macOS menu bar app backed by a small Cloudflare Worker proxy. The Swift app handles all local UX, permissions, audio/screen capture, and playback; the Worker keeps third-party API keys out of the app binary and exposes a few thin API endpoints.

2. **Local app boundary**
   The main target is `leanring-buddy` in Xcode, built as a menu bar utility rather than a normal dock app. The UI is centered around `NSPanel`-style floating windows: a compact companion/control panel and a transparent overlay window used for cursor guidance.

3. **Primary controller**
   `CompanionManager.swift` appears to be the central state/orchestration layer. It coordinates dictation, transcription, Claude requests, TTS playback, screen capture, overlay behavior, and app state transitions, making it the key architectural hub for diagramming.

4. **Voice input pipeline**
   Push-to-talk and dictation are handled by `BuddyDictationManager.swift`, `BuddyTranscriptionProvider.swift`, and provider-specific implementations for Apple, AssemblyAI, and OpenAI. The app captures microphone audio locally, then either transcribes locally/on-device or streams audio to a remote transcription service depending on the selected path.

5. **Remote transcription flow**
   For real-time transcription, the app uses AssemblyAI and OpenAI-specific adapters. The README indicates AssemblyAI token provisioning is proxied through the Worker, while audio streams over a websocket-based flow. This makes transcription one of the main networked subsystems in the app.

6. **Screen understanding**
   The app can inspect the user’s current UI by capturing the screen and locating elements. `CompanionScreenCaptureUtility.swift` and `ElementLocationDetector.swift` suggest a capture-and-analysis pipeline that produces screenshots and coordinates for the model, enabling the “point at stuff” behavior.

7. **LLM interaction**
   `ClaudeAPI.swift` is the chat/assistant client. The app sends the transcript plus screenshot/context to Claude using streaming SSE, then consumes streamed responses in real time. Claude is also allowed to emit point tags like `[POINT:x,y:label:screenN]`, which drive the cursor overlay.

8. **TTS and audio feedback**
   `ElevenLabsTTSClient.swift` handles voice playback for the assistant response. The README and file layout suggest the text response is converted to speech via ElevenLabs and played back locally, with supporting audio assets like `enter.mp3`, `eshop.mp3`, and `ff.mp3` likely used for UI or interaction cues.

9. **Cursor/overlay system**
   `OverlayWindow.swift`, `CompanionResponseOverlay.swift`, and `WindowPositionManager.swift` implement the visual guidance layer. This subsystem renders the movable blue cursor/overlay next to or over the user’s screen so the assistant can visually direct attention to specific UI locations.

10. **Global interaction and permissions**
   `GlobalPushToTalkShortcutMonitor.swift` and the entitlement/plist files indicate the app relies on accessibility and screen-recording permissions for global hotkeys and screen access. This is an important architecture boundary: the app depends on macOS security capabilities to function as an always-available assistant.

11. **Cloudflare Worker proxy**
   The `worker/` package is a small TypeScript Cloudflare Worker with three proxy responsibilities: Claude chat, ElevenLabs TTS, and AssemblyAI token issuance. It is the network boundary between the macOS app and external AI services, and it holds the sensitive API keys via Worker secrets.

12. **External services**
   The system integrates three main third-party services: Anthropic Claude for reasoning/chat, AssemblyAI for live transcription, and ElevenLabs for speech synthesis. The Worker mediates all of them, so the app itself only talks to one proxy host instead of directly to each vendor.

13. **Supporting app infrastructure**
   `AppBundleConfiguration.swift`, `ClickyAnalytics.swift`, and `DesignSystem.swift` indicate supporting layers for app configuration, telemetry, and shared UI styling. These are cross-cutting concerns rather than primary product flows.

14. **Repository composition**
   The repo is split into a macOS client, a Cloudflare Worker backend, and lightweight scripts. `scripts/release.sh` and `appcast.xml` suggest there is also a release/update distribution path, but the core runtime architecture is the client-plus-proxy model described above.

Main entry points: [`leanring-buddy/CompanionManager.swift`](https://github.com/themanojdesai/clicky/blob/main/leanring-buddy/CompanionManager.swift) (Companion core), [`worker/src/index.ts`](https://github.com/themanojdesai/clicky/blob/main/worker/src/index.ts) (Worker proxy), [`leanring-buddy/leanring_buddyApp.swift`](https://github.com/themanojdesai/clicky/blob/main/leanring-buddy/leanring_buddyApp.swift) (App entry), [`leanring-buddy/BuddyTranscriptionProvider.swift`](https://github.com/themanojdesai/clicky/blob/main/leanring-buddy/BuddyTranscriptionProvider.swift) (Transcription API)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_client["macOS app"]
  node_app(("App entry<br/>swift app"))
  node_bundlecfg["Bundle config"]
  node_companion{{"Companion core<br/>orchestrator"}}
  node_panel["Panel UI<br/>ui panel"]
  node_menu["Menu bar window<br/>window manager"]
  node_overlay["Overlay system<br/>overlay ui"]
  node_respovl["Response overlay<br/>overlay render"]
  node_windowpos["Window positioning<br/>layout"]
  node_dictation{{"Dictation<br/>voice input"}}
  node_transcribe["Transcription API<br/>provider interface"]
  node_applestt["Apple STT<br/>local transcription"]
  node_asr["AssemblyAI stream<br/>remote transcription"]
  node_openaiasr["OpenAI STT<br/>remote transcription"]
  node_screen["Screen capture<br/>screen analysis"]
  node_locator["Element locator<br/>analysis"]
  node_claude["Claude client<br/>llm client<br/>[ClaudeAPI.swift]"]
  node_tts["TTS playback<br/>speech synthesis"]
  node_audio["Audio support<br/>audio utils"]
  node_hotkey["Global hotkey<br/>accessibility"]
  node_analytics["Analytics<br/>telemetry"]
  node_design["Design system<br/>ui styling<br/>[DesignSystem.swift]"]
end

subgraph group_worker["Cloudflare Worker"]
  node_workerapi{{"Worker proxy<br/>edge proxy<br/>[index.ts]"}}
end

subgraph group_release["Release tooling"]
  node_release["Release script<br/>release tooling<br/>[release.sh]"]
end

node_app -->|"boots"| node_companion
node_app -->|"creates"| node_menu
node_app -->|"shows"| node_panel
node_app -->|"opens"| node_overlay
node_companion -->|"controls"| node_dictation
node_companion -->|"captures"| node_screen
node_companion -->|"queries"| node_claude
node_companion -->|"speaks"| node_tts
node_companion -->|"guides"| node_respovl
node_companion -->|"positions"| node_windowpos
node_companion -.->|"tracks"| node_analytics
node_companion -.->|"styles"| node_design
node_dictation -->|"listens"| node_hotkey
node_dictation -->|"selects"| node_transcribe
node_transcribe -->|"implements"| node_applestt
node_transcribe -->|"implements"| node_asr
node_transcribe -->|"implements"| node_openaiasr
node_screen -->|"analyzes"| node_locator
node_claude -->|"proxies via"| node_workerapi
node_tts -->|"proxies via"| node_workerapi
node_asr -->|"token via"| node_workerapi
node_workerapi -->|"forwards"| node_claude
node_workerapi -->|"forwards"| node_tts
node_workerapi -->|"mints token"| node_asr
node_overlay -->|"renders"| node_respovl
node_menu -->|"presents"| node_panel
node_windowpos -->|"places"| node_overlay
node_audio -.->|"supports"| node_tts
node_release -.->|"deploys"| node_workerapi

click node_app "https://github.com/themanojdesai/clicky/blob/main/leanring-buddy/leanring_buddyApp.swift"
click node_bundlecfg "https://github.com/themanojdesai/clicky/blob/main/leanring-buddy/AppBundleConfiguration.swift"
click node_companion "https://github.com/themanojdesai/clicky/blob/main/leanring-buddy/CompanionManager.swift"
click node_panel "https://github.com/themanojdesai/clicky/blob/main/leanring-buddy/CompanionPanelView.swift"
click node_menu "https://github.com/themanojdesai/clicky/blob/main/leanring-buddy/MenuBarPanelManager.swift"
click node_overlay "https://github.com/themanojdesai/clicky/blob/main/leanring-buddy/OverlayWindow.swift"
click node_respovl "https://github.com/themanojdesai/clicky/blob/main/leanring-buddy/CompanionResponseOverlay.swift"
click node_windowpos "https://github.com/themanojdesai/clicky/blob/main/leanring-buddy/WindowPositionManager.swift"
click node_dictation "https://github.com/themanojdesai/clicky/blob/main/leanring-buddy/BuddyDictationManager.swift"
click node_transcribe "https://github.com/themanojdesai/clicky/blob/main/leanring-buddy/BuddyTranscriptionProvider.swift"
click node_applestt "https://github.com/themanojdesai/clicky/blob/main/leanring-buddy/AppleSpeechTranscriptionProvider.swift"
click node_asr "https://github.com/themanojdesai/clicky/blob/main/leanring-buddy/AssemblyAIStreamingTranscriptionProvider.swift"
click node_openaiasr "https://github.com/themanojdesai/clicky/blob/main/leanring-buddy/OpenAIAudioTranscriptionProvider.swift"
click node_screen "https://github.com/themanojdesai/clicky/blob/main/leanring-buddy/CompanionScreenCaptureUtility.swift"
click node_locator "https://github.com/themanojdesai/clicky/blob/main/leanring-buddy/ElementLocationDetector.swift"
click node_claude "https://github.com/themanojdesai/clicky/blob/main/leanring-buddy/ClaudeAPI.swift"
click node_tts "https://github.com/themanojdesai/clicky/blob/main/leanring-buddy/ElevenLabsTTSClient.swift"
click node_audio "https://github.com/themanojdesai/clicky/blob/main/leanring-buddy/BuddyAudioConversionSupport.swift"
click node_hotkey "https://github.com/themanojdesai/clicky/blob/main/leanring-buddy/GlobalPushToTalkShortcutMonitor.swift"
click node_analytics "https://github.com/themanojdesai/clicky/blob/main/leanring-buddy/ClickyAnalytics.swift"
click node_design "https://github.com/themanojdesai/clicky/blob/main/leanring-buddy/DesignSystem.swift"
click node_workerapi "https://github.com/themanojdesai/clicky/blob/main/worker/src/index.ts"
click node_release "https://github.com/themanojdesai/clicky/blob/main/scripts/release.sh"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_app,node_bundlecfg,node_companion,node_panel,node_menu,node_overlay,node_respovl,node_windowpos,node_dictation,node_transcribe,node_applestt,node_asr,node_openaiasr,node_screen,node_locator,node_claude,node_tts,node_audio,node_hotkey,node_analytics,node_design toneBlue
class node_workerapi toneAmber
class node_release toneMint
```

## Components

### macOS app

Swift menu bar client and local interaction pipeline

- [App entry](https://github.com/themanojdesai/clicky/blob/main/leanring-buddy/leanring_buddyApp.swift): swift app · Menu bar app bootstrap and lifecycle · `leanring-buddy/leanring_buddyApp.swift`
- [Bundle config](https://github.com/themanojdesai/clicky/blob/main/leanring-buddy/AppBundleConfiguration.swift): config · App bundle and runtime configuration · `leanring-buddy/AppBundleConfiguration.swift`
- [Companion core](https://github.com/themanojdesai/clicky/blob/main/leanring-buddy/CompanionManager.swift): orchestrator · Central coordinator for dictation, screen capture, overlays, TTS, and state · `leanring-buddy/CompanionManager.swift`
- [Panel UI](https://github.com/themanojdesai/clicky/blob/main/leanring-buddy/CompanionPanelView.swift): ui panel · Floating companion/control panel · `leanring-buddy/CompanionPanelView.swift`
- [Menu bar window](https://github.com/themanojdesai/clicky/blob/main/leanring-buddy/MenuBarPanelManager.swift): window manager · Menu bar panel/window presentation · `leanring-buddy/MenuBarPanelManager.swift`
- [Overlay system](https://github.com/themanojdesai/clicky/blob/main/leanring-buddy/OverlayWindow.swift): overlay ui · Transparent cursor guidance and response overlay · `leanring-buddy/OverlayWindow.swift`
- [Response overlay](https://github.com/themanojdesai/clicky/blob/main/leanring-buddy/CompanionResponseOverlay.swift): overlay render · Renders assistant point/cursor guidance · `leanring-buddy/CompanionResponseOverlay.swift`
- [Window positioning](https://github.com/themanojdesai/clicky/blob/main/leanring-buddy/WindowPositionManager.swift): layout · Positions floating windows and overlay elements · `leanring-buddy/WindowPositionManager.swift`
- [Dictation](https://github.com/themanojdesai/clicky/blob/main/leanring-buddy/BuddyDictationManager.swift): voice input · Push-to-talk capture and transcription coordination · `leanring-buddy/BuddyDictationManager.swift`
- [Transcription API](https://github.com/themanojdesai/clicky/blob/main/leanring-buddy/BuddyTranscriptionProvider.swift): provider interface · Shared abstraction for speech-to-text providers · `leanring-buddy/BuddyTranscriptionProvider.swift`
- [Apple STT](https://github.com/themanojdesai/clicky/blob/main/leanring-buddy/AppleSpeechTranscriptionProvider.swift): local transcription · On-device speech transcription path · `leanring-buddy/AppleSpeechTranscriptionProvider.swift`
- [AssemblyAI stream](https://github.com/themanojdesai/clicky/blob/main/leanring-buddy/AssemblyAIStreamingTranscriptionProvider.swift): remote transcription · Streaming transcription provider for live audio · `leanring-buddy/AssemblyAIStreamingTranscriptionProvider.swift`
- [OpenAI STT](https://github.com/themanojdesai/clicky/blob/main/leanring-buddy/OpenAIAudioTranscriptionProvider.swift): remote transcription · OpenAI transcription adapter · `leanring-buddy/OpenAIAudioTranscriptionProvider.swift`
- [Screen capture](https://github.com/themanojdesai/clicky/blob/main/leanring-buddy/CompanionScreenCaptureUtility.swift): screen analysis · Captures screenshots for context and pointing · `leanring-buddy/CompanionScreenCaptureUtility.swift`
- [Element locator](https://github.com/themanojdesai/clicky/blob/main/leanring-buddy/ElementLocationDetector.swift): analysis · Finds UI elements and coordinates in captured screens · `leanring-buddy/ElementLocationDetector.swift`
- [Claude client](https://github.com/themanojdesai/clicky/blob/main/leanring-buddy/ClaudeAPI.swift): llm client · Streams prompts and responses from Claude · `leanring-buddy/ClaudeAPI.swift`
- [TTS playback](https://github.com/themanojdesai/clicky/blob/main/leanring-buddy/ElevenLabsTTSClient.swift): speech synthesis · ElevenLabs voice synthesis client · `leanring-buddy/ElevenLabsTTSClient.swift`
- [Audio support](https://github.com/themanojdesai/clicky/blob/main/leanring-buddy/BuddyAudioConversionSupport.swift): audio utils · Helpers for audio conversion and playback inputs · `leanring-buddy/BuddyAudioConversionSupport.swift`
- [Global hotkey](https://github.com/themanojdesai/clicky/blob/main/leanring-buddy/GlobalPushToTalkShortcutMonitor.swift): accessibility · Push-to-talk shortcut monitor and global input hook · `leanring-buddy/GlobalPushToTalkShortcutMonitor.swift`
- [Analytics](https://github.com/themanojdesai/clicky/blob/main/leanring-buddy/ClickyAnalytics.swift): telemetry · Usage and event tracking · `leanring-buddy/ClickyAnalytics.swift`
- [Design system](https://github.com/themanojdesai/clicky/blob/main/leanring-buddy/DesignSystem.swift): ui styling · Shared visual styling and UI constants · `leanring-buddy/DesignSystem.swift`

### Cloudflare Worker

Thin proxy for third-party AI services

- [Worker proxy](https://github.com/themanojdesai/clicky/blob/main/worker/src/index.ts): edge proxy · Cloudflare Worker entrypoint for vendor API mediation · `worker/src/index.ts`

### Release tooling

Distribution and update helpers

- [Release script](https://github.com/themanojdesai/clicky/blob/main/scripts/release.sh): release tooling · Build and publish helper for distribution · `scripts/release.sh`

## Connections

- App entry → Companion core: boots · App lifecycle starts the main coordinator
- App entry → Menu bar window: creates · Bootstraps menu bar presentation
- App entry → Panel UI: shows · Displays companion UI
- App entry → Overlay system: opens · Initializes overlay windows
- Companion core → Dictation: controls · Starts and stops voice capture
- Companion core → Screen capture: captures · Requests screenshots and screen context
- Companion core → Claude client: queries · Sends transcript and context to the model
- Companion core → TTS playback: speaks · Converts assistant text to audio
- Companion core → Response overlay: guides · Feeds point tags and guidance to overlay rendering
- Companion core → Window positioning: positions · Coordinates floating UI placement
- Companion core → Analytics: tracks · Emits app usage telemetry
- Companion core → Design system: styles · Uses shared UI styling
- Dictation → Global hotkey: listens · Uses global push-to-talk shortcuts
- Dictation → Transcription API: selects · Routes audio to a transcription provider
- Transcription API → Apple STT: implements · Local transcription path
- Transcription API → AssemblyAI stream: implements · Streaming AssemblyAI path
- Transcription API → OpenAI STT: implements · OpenAI transcription path
- Screen capture → Element locator: analyzes · Produces UI element locations from screenshots
- Claude client → Worker proxy: proxies via · Uses the Worker for vendor API access
- TTS playback → Worker proxy: proxies via · Routes ElevenLabs requests through the Worker
- AssemblyAI stream → Worker proxy: token via · Obtains AssemblyAI token through the Worker
- Worker proxy → Claude client: forwards · Proxies Claude chat requests
- Worker proxy → TTS playback: forwards · Proxies ElevenLabs TTS requests
- Worker proxy → AssemblyAI stream: mints token · Issues AssemblyAI access token
- Overlay system → Response overlay: renders · Hosts the visual guidance overlay
- Menu bar window → Panel UI: presents · Displays the companion panel from menu-bar UI
- Window positioning → Overlay system: places · Computes overlay placement
- Audio support → TTS playback: supports · Provides audio conversion and playback support
- Release script → Worker proxy: deploys · Supports release/deployment flow
