How farzaa/clicky works
1. Overall shape
- This repo is a macOS desktop app plus a small Cloudflare Worker backend. The app is the product; the Worker is a thin proxy layer that keeps third-party API keys out of the client binary.
2. Primary runtime boundary
- leanring-buddy/ is the Swift app target.
- worker/ is a TypeScript Cloudflare Worker deployed separately.
- The app never talks directly to Anthropic, AssemblyAI, or ElevenLabs; it calls the Worker, which forwards requests to those vendors.
3. macOS app shell
- The app is a menu bar utility, not a normal docked app.
- leanring_buddyApp.swift, MenuBarPanelManager.swift, and CompanionPanelView.swift coordinate the app lifecycle and the menu-bar UI.
- OverlayWindow.swift provides the transparent/full-screen overlay used to draw attention to UI elements near the cursor.
4. Central orchestration
- CompanionManager.swift is the main state machine and coordination layer.
- It ties together transcription, Claude chat, screen capture, TTS playback, and overlay behavior.
- Most user-facing flows converge here, which makes it the best single point for understanding request/response sequencing.
5. Voice input pipeline
- Push-to-talk is handled by GlobalPushToTalkShortcutMonitor.swift, BuddyDictationManager.swift, and BuddyAudioConversionSupport.swift.
- Audio is captured from the microphone, converted as needed, and streamed to transcription services.
- The app supports more than one transcription provider: AssemblyAIStreamingTranscriptionProvider.swift, OpenAIAudioTranscriptionProvider.swift, and AppleSpeechTranscriptionProvider.swift, all conforming to BuddyTranscriptionProvider.swift.
6. Transcription backend flow
- For real-time dictation, the app requests a short-lived AssemblyAI token from the Worker and then streams audio to AssemblyAI.
- This is a good example of the repo’s trust boundary: sensitive credentials stay server-side, while the client only receives scoped runtime tokens or proxied responses.
7. Claude + response generation
- ClaudeAPI.swift sends the transcript plus any captured screen context to Claude through the Worker’s /chat route.
- Claude responses can include structured point-to-screen instructions, which the app interprets to move or highlight attention on the display.
- ElementLocationDetector.swift and CompanionScreenCaptureUtility.swift help translate screen information into actionable coordinates.
8. Speech output
- ElevenLabsTTSClient.swift turns Claude’s response text into audio via the Worker’s /tts route.
- The app includes local sound assets like enter.mp3, eshop.mp3, and ff.mp3 for UI/audio cues in addition to streamed TTS.
9. Screen interaction and pointing
- OverlayWindow.swift, CompanionResponseOverlay.swift, and WindowPositionManager.swift manage visual positioning relative to the cursor and display geometry.
- The overlay subsystem is separate from the main panel so the app can present guidance without taking over the whole desktop.
- Multi-monitor awareness is implied by the screen-location handling in the response format.
10. Analytics, configuration, and distribution
- ClickyAnalytics.swift suggests lightweight product telemetry.
- AppBundleConfiguration.swift and Info.plist define app-level packaging and runtime settings.
- appcast.xml and scripts/release.sh indicate an update/release workflow, likely for app distribution outside the App Store.
11. Worker structure
- worker/src/index.ts is the Worker entrypoint; wrangler.toml defines deployment configuration.
- The Worker exposes three main proxy responsibilities: Claude chat, ElevenLabs TTS, and AssemblyAI transcription-token issuance.
- Its job is deliberately small: route, authenticate with secret keys, and return vendor responses to the macOS client.
12. Operational dependencies
- The system depends on macOS permissions for microphone, accessibility, and screen recording / screen content access.
- At runtime the app relies on ScreenCaptureKit-style screen access, a global keyboard shortcut, and network connectivity to the Worker and downstream AI services.
13. What an architecture diagram should show
- A left-to-right flow works well: user input on the Mac app, optional screen capture, transcription, Claude reasoning, TTS output, and overlay feedback.
- Place the Cloudflare Worker as the security boundary in the middle, with arrows from the app to the Worker and from the Worker to Anthropic, AssemblyAI, and ElevenLabs.
Main entry points: CompanionManager.swift Companion manager · index.ts Worker entry · BuddyDictationManager.swift Dictation flow · BuddyTranscriptionProvider.swift Transcription API
Components 25
macOS App
Swift menu-bar desktop client
- App entry · Launches the menu-bar app lifecycle
leanring-buddy/leanring_buddyApp.swift - Menu bar · Menu-bar panel and companion UI shell
leanring-buddy/MenuBarPanelManager.swift - Companion panel · Primary visible companion panel
leanring-buddy/CompanionPanelView.swift - Companion manager · Main state machine coordinating the user flow
leanring-buddy/CompanionManager.swift - Overlay window · Transparent attention overlay near the cursor
leanring-buddy/OverlayWindow.swift - Response overlay · Renders point-to-screen guidance and highlights
leanring-buddy/CompanionResponseOverlay.swift - Window position · Positions panels and overlays across displays
leanring-buddy/WindowPositionManager.swift - Dictation flow · Captures and prepares microphone audio for transcription
leanring-buddy/BuddyDictationManager.swift - Push-to-talk · Global keyboard shortcut for voice capture
leanring-buddy/GlobalPushToTalkShortcutMonitor.swift - Audio convert · Audio conversion and format support
leanring-buddy/BuddyAudioConversionSupport.swift - Transcription API · Common interface for transcription providers
leanring-buddy/BuddyTranscriptionProvider.swift - AssemblyAI stream · Real-time streaming transcription provider
leanring-buddy/AssemblyAIStreamingTranscriptionProvider.swift - OpenAI speech · Alternative audio transcription provider
leanring-buddy/OpenAIAudioTranscriptionProvider.swift - Apple speech · Local Apple speech transcription provider
leanring-buddy/AppleSpeechTranscriptionProvider.swift - Claude client · Sends transcript and screen context for reasoning
leanring-buddy/ClaudeAPI.swift - TTS client · Turns responses into audio playback
leanring-buddy/ElevenLabsTTSClient.swift - Screen capture · Collects screen context for prompts and pointing
leanring-buddy/CompanionScreenCaptureUtility.swift - Element lookup · Maps screen content to actionable coordinates
leanring-buddy/ElementLocationDetector.swift - Analytics · Lightweight product analytics
leanring-buddy/ClickyAnalytics.swift
Cloudflare Worker
Thin proxy and token boundary
- Worker entry · Cloudflare Worker request router
worker/src/index.ts - Worker config · Deployment and bindings configuration
worker/wrangler.toml
External Services
Vendor APIs and system inputs
- Anthropic · Claude chat backend
- AssemblyAI · Streaming transcription backend
- ElevenLabs · Text-to-speech backend
- macOS permissions · Microphone, accessibility, and screen recording access
Connections 25
- App entry → Menu bar: boots UI · Launches the menu-bar shell
- Menu bar → Companion panel: shows · Presents the companion panel
- Menu bar → Companion manager: owns · Creates and coordinates app state
- Companion manager → Push-to-talk: listens · Waits for push-to-talk input
- Companion manager → Dictation flow: starts capture · Begins microphone capture and transcription flow
- Dictation flow → Audio convert: normalizes · Prepares audio for providers
- Dictation flow → Transcription API: uses · Routes audio through the provider interface
- Transcription API → AssemblyAI stream: implements · Streaming AssemblyAI provider
- Transcription API → OpenAI speech: implements · OpenAI transcription provider
- Transcription API → Apple speech: implements · Apple speech provider
- Companion manager → Screen capture: captures context · Collects screen context for reasoning
- Screen capture → Element lookup: analyzes · Turns capture into screen targets
- Companion manager → Claude client: sends prompt · Combines transcript and screen context for Claude
- Companion manager → TTS client: speaks reply · Converts response text to audio
- Companion manager → Overlay window: highlights · Shows guidance overlay near cursor
- Companion manager → Response overlay: renders response · Displays point-to-screen instructions
- Companion manager → Window position: positions · Keeps windows aligned across displays
- Companion manager → Analytics: logs · Records lightweight telemetry
- Claude client → Worker entry: proxied via · Sends chat requests through the worker
- TTS client → Worker entry: proxied via · Requests TTS through the worker
- Dictation flow → Worker entry: token request · Fetches short-lived transcription token
- Worker entry → Anthropic: forwards · Proxies chat completion calls
- Worker entry → AssemblyAI: issues token · Returns scoped AssemblyAI token
- Worker entry → ElevenLabs: forwards · Proxies text-to-speech calls
- Companion manager → macOS permissions: depends on · Requires OS-level capture and shortcut permissions