How bgykanishka/butler works
Butler is a privacy-first macOS meeting assistant. The main workflow is to start a session, capture microphone and system audio, transcribe speech, retain recent conversation context, evaluate when to answer, and stream a local-model response to the UI. Separate branches support project-codebase context and interactive screen questions; permissions and model setup support these workflows. The README documents several connections not present in the sampled source, so those are represented only where it explicitly describes them; detailed session wiring is not sampled.
Main entry points: MenuBarManager.swift Menu bar and UI · SessionCoordinator.swift Session coordinator · LocalLLMEngine.swift Local language model · ResponseGenerator.swift Response evaluation
Components 20
App and session
- Menu bar and UI
butler/UI/MenuBar/MenuBarManager.swift - Session coordinator
butler/Session/SessionCoordinator.swift
Audio and speech
- Microphone capture
butler/Audio/MicrophoneCaptureService.swift - System audio capture
butler/Audio/SystemAudioCaptureService.swift - Audio ring buffer
butler/Audio/AudioRingBuffer.swift - Voice activity detection
butler/Audio/VoiceActivityDetector.swift - Whisper transcription
butler/Speech/WhisperEngine.swift - Transcript assembly
butler/Speech/TranscriptAssembler.swift
Response intelligence
- Response evaluation
butler/Intelligence/ResponseGenerator.swift - Prompt construction
butler/Intelligence/PromptBuilder.swift - Local language model
butler/LLM/LocalLLMEngine.swift
Context and vision
- Conversation context
butler/Context/ContextManager.swift - Project context
butler/Context/ProjectContextManager.swift - Codebase analyzer
butler/Context/ProjectAnalyzer.swift - Interactive screen capture
butler/FutureVision/InteractiveCaptureService.swift - Vision processing
butler/FutureVision/OCRService.swift
Models and access
- Permission gateway
butler/Permissions/PermissionsGateway.swift - Model management
butler/LLM/ModelManager.swift - Model downloads
butler/LLM/ModelDownloader.swift
Other components
- Meeting user
Connections 27
- Meeting user → Menu bar and UI: controls
- Menu bar and UI → Session coordinator: starts session
- Menu bar and UI → Session coordinator: triggers screen analysis
- Menu bar and UI → Permission gateway: requests access
- Session coordinator → Microphone capture: coordinates capture
- Session coordinator → System audio capture: coordinates capture
- Microphone capture → Audio ring buffer: supplies samples
- System audio capture → Audio ring buffer: supplies samples
- Audio ring buffer → Voice activity detection: feeds audio
- Voice activity detection → Whisper transcription: gates speech
- Whisper transcription → Transcript assembly: emits transcripts
- Transcript assembly → Response evaluation: supplies segments
- Response evaluation → Conversation context: records turns
- Conversation context → Response evaluation: provides context
- Response evaluation → Prompt construction: requests prompt
- Prompt construction → Local language model: supplies prompt
- Local language model → Menu bar and UI: streams tokens
- Project context → Codebase analyzer: analyzes repositories
- Project context → Prompt construction: supplies project summary
- Project context → Local language model: saves project state
- Menu bar and UI → Interactive screen capture: starts capture
- Interactive screen capture → Vision processing: provides image
- Vision processing → Local language model: supplies visual input
- Session coordinator → Permission gateway: checks access
- Model management → Whisper transcription: validates model
- Model management → Local language model: validates model
- Model downloads → Meeting user: downloads models