# bgykanishka/butler architecture

> Architecture diagram of the GitHub repository bgykanishka/butler, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-09-27). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/bgykanishka/butler
- Repository: https://github.com/bgykanishka/butler

## Overview

Butler is a privacy-first macOS meeting assistant. The main workflow is to start a session, capture microphone and system audio, transcribe speech, retain recent conversation context, evaluate when to answer, and stream a local-model response to the UI. Separate branches support project-codebase context and interactive screen questions; permissions and model setup support these workflows. The README documents several connections not present in the sampled source, so those are represented only where it explicitly describes them; detailed session wiring is not sampled.

Main entry points: [`butler/UI/MenuBar/MenuBarManager.swift`](https://github.com/bgykanishka/butler/blob/main/butler/UI/MenuBar/MenuBarManager.swift) (Menu bar and UI), [`butler/Session/SessionCoordinator.swift`](https://github.com/bgykanishka/butler/blob/main/butler/Session/SessionCoordinator.swift) (Session coordinator), [`butler/LLM/LocalLLMEngine.swift`](https://github.com/bgykanishka/butler/blob/main/butler/LLM/LocalLLMEngine.swift) (Local language model), [`butler/Intelligence/ResponseGenerator.swift`](https://github.com/bgykanishka/butler/blob/main/butler/Intelligence/ResponseGenerator.swift) (Response evaluation)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_app["App and session"]
  node_ui["Menu bar and UI"]
  node_session["Session coordinator"]
end

subgraph group_audio["Audio and speech"]
  node_mic["Microphone capture"]
  node_systemaudio["System audio capture"]
  node_ring["Audio ring buffer"]
  node_vad["Voice activity detection"]
  node_whisper["Whisper transcription"]
  node_assembler["Transcript assembly"]
end

subgraph group_intelligence["Response intelligence"]
  node_response["Response evaluation"]
  node_prompt["Prompt construction"]
  node_llm["Local language model"]
end

subgraph group_context["Context and vision"]
  node_conversation["Conversation context"]
  node_project["Project context"]
  node_analyzer["Codebase analyzer"]
  node_capture["Interactive screen capture"]
  node_vision["Vision processing<br/>[OCRService.swift]"]
end

subgraph group_models["Models and access"]
  node_permissions["Permission gateway"]
  node_modelmanager["Model management<br/>[ModelManager.swift]"]
  node_downloader["Model downloads"]
end

node_user(("Meeting user"))

node_user -->|"controls"| node_ui
node_ui -->|"starts session"| node_session
node_ui -->|"triggers screen analysis"| node_session
node_ui -->|"requests access"| node_permissions
node_session -->|"coordinates capture"| node_mic
node_session -->|"coordinates capture"| node_systemaudio
node_mic -->|"supplies samples"| node_ring
node_systemaudio -->|"supplies samples"| node_ring
node_ring -->|"feeds audio"| node_vad
node_vad -->|"gates speech"| node_whisper
node_whisper -->|"emits transcripts"| node_assembler
node_assembler -->|"supplies segments"| node_response
node_response -->|"records turns"| node_conversation
node_conversation -->|"provides context"| node_response
node_response -->|"requests prompt"| node_prompt
node_prompt -->|"supplies prompt"| node_llm
node_llm -->|"streams tokens"| node_ui
node_project -->|"analyzes repositories"| node_analyzer
node_project -->|"supplies project summary"| node_prompt
node_project -->|"saves project state"| node_llm
node_ui -->|"starts capture"| node_capture
node_capture -.->|"provides image"| node_vision
node_vision -.->|"supplies visual input"| node_llm
node_session -->|"checks access"| node_permissions
node_modelmanager -->|"validates model"| node_whisper
node_modelmanager -->|"validates model"| node_llm
node_downloader -->|"downloads models"| node_user

click node_ui "https://github.com/bgykanishka/butler/blob/main/butler/UI/MenuBar/MenuBarManager.swift"
click node_session "https://github.com/bgykanishka/butler/blob/main/butler/Session/SessionCoordinator.swift"
click node_mic "https://github.com/bgykanishka/butler/blob/main/butler/Audio/MicrophoneCaptureService.swift"
click node_systemaudio "https://github.com/bgykanishka/butler/blob/main/butler/Audio/SystemAudioCaptureService.swift"
click node_ring "https://github.com/bgykanishka/butler/blob/main/butler/Audio/AudioRingBuffer.swift"
click node_vad "https://github.com/bgykanishka/butler/blob/main/butler/Audio/VoiceActivityDetector.swift"
click node_whisper "https://github.com/bgykanishka/butler/blob/main/butler/Speech/WhisperEngine.swift"
click node_assembler "https://github.com/bgykanishka/butler/blob/main/butler/Speech/TranscriptAssembler.swift"
click node_response "https://github.com/bgykanishka/butler/blob/main/butler/Intelligence/ResponseGenerator.swift"
click node_prompt "https://github.com/bgykanishka/butler/blob/main/butler/Intelligence/PromptBuilder.swift"
click node_llm "https://github.com/bgykanishka/butler/blob/main/butler/LLM/LocalLLMEngine.swift"
click node_conversation "https://github.com/bgykanishka/butler/blob/main/butler/Context/ContextManager.swift"
click node_project "https://github.com/bgykanishka/butler/blob/main/butler/Context/ProjectContextManager.swift"
click node_analyzer "https://github.com/bgykanishka/butler/blob/main/butler/Context/ProjectAnalyzer.swift"
click node_capture "https://github.com/bgykanishka/butler/blob/main/butler/FutureVision/InteractiveCaptureService.swift"
click node_vision "https://github.com/bgykanishka/butler/blob/main/butler/FutureVision/OCRService.swift"
click node_permissions "https://github.com/bgykanishka/butler/blob/main/butler/Permissions/PermissionsGateway.swift"
click node_modelmanager "https://github.com/bgykanishka/butler/blob/main/butler/LLM/ModelManager.swift"
click node_downloader "https://github.com/bgykanishka/butler/blob/main/butler/LLM/ModelDownloader.swift"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_ui,node_session,node_user toneBlue
class node_mic,node_systemaudio,node_ring,node_vad,node_whisper,node_assembler toneAmber
class node_response,node_prompt,node_llm toneMint
class node_conversation,node_project,node_analyzer,node_capture,node_vision toneRose
class node_permissions,node_modelmanager,node_downloader toneIndigo
```

## Components

### App and session

- [Menu bar and UI](https://github.com/bgykanishka/butler/blob/main/butler/UI/MenuBar/MenuBarManager.swift): `butler/UI/MenuBar/MenuBarManager.swift`
- [Session coordinator](https://github.com/bgykanishka/butler/blob/main/butler/Session/SessionCoordinator.swift): `butler/Session/SessionCoordinator.swift`

### Audio and speech

- [Microphone capture](https://github.com/bgykanishka/butler/blob/main/butler/Audio/MicrophoneCaptureService.swift): `butler/Audio/MicrophoneCaptureService.swift`
- [System audio capture](https://github.com/bgykanishka/butler/blob/main/butler/Audio/SystemAudioCaptureService.swift): `butler/Audio/SystemAudioCaptureService.swift`
- [Audio ring buffer](https://github.com/bgykanishka/butler/blob/main/butler/Audio/AudioRingBuffer.swift): `butler/Audio/AudioRingBuffer.swift`
- [Voice activity detection](https://github.com/bgykanishka/butler/blob/main/butler/Audio/VoiceActivityDetector.swift): `butler/Audio/VoiceActivityDetector.swift`
- [Whisper transcription](https://github.com/bgykanishka/butler/blob/main/butler/Speech/WhisperEngine.swift): `butler/Speech/WhisperEngine.swift`
- [Transcript assembly](https://github.com/bgykanishka/butler/blob/main/butler/Speech/TranscriptAssembler.swift): `butler/Speech/TranscriptAssembler.swift`

### Response intelligence

- [Response evaluation](https://github.com/bgykanishka/butler/blob/main/butler/Intelligence/ResponseGenerator.swift): `butler/Intelligence/ResponseGenerator.swift`
- [Prompt construction](https://github.com/bgykanishka/butler/blob/main/butler/Intelligence/PromptBuilder.swift): `butler/Intelligence/PromptBuilder.swift`
- [Local language model](https://github.com/bgykanishka/butler/blob/main/butler/LLM/LocalLLMEngine.swift): `butler/LLM/LocalLLMEngine.swift`

### Context and vision

- [Conversation context](https://github.com/bgykanishka/butler/blob/main/butler/Context/ContextManager.swift): `butler/Context/ContextManager.swift`
- [Project context](https://github.com/bgykanishka/butler/blob/main/butler/Context/ProjectContextManager.swift): `butler/Context/ProjectContextManager.swift`
- [Codebase analyzer](https://github.com/bgykanishka/butler/blob/main/butler/Context/ProjectAnalyzer.swift): `butler/Context/ProjectAnalyzer.swift`
- [Interactive screen capture](https://github.com/bgykanishka/butler/blob/main/butler/FutureVision/InteractiveCaptureService.swift): `butler/FutureVision/InteractiveCaptureService.swift`
- [Vision processing](https://github.com/bgykanishka/butler/blob/main/butler/FutureVision/OCRService.swift): `butler/FutureVision/OCRService.swift`

### Models and access

- [Permission gateway](https://github.com/bgykanishka/butler/blob/main/butler/Permissions/PermissionsGateway.swift): `butler/Permissions/PermissionsGateway.swift`
- [Model management](https://github.com/bgykanishka/butler/blob/main/butler/LLM/ModelManager.swift): `butler/LLM/ModelManager.swift`
- [Model downloads](https://github.com/bgykanishka/butler/blob/main/butler/LLM/ModelDownloader.swift): `butler/LLM/ModelDownloader.swift`

### Other components

- **Meeting user**

## Connections

- Meeting user → Menu bar and UI: controls
- Menu bar and UI → Session coordinator: starts session
- Menu bar and UI → Session coordinator: triggers screen analysis
- Menu bar and UI → Permission gateway: requests access
- Session coordinator → Microphone capture: coordinates capture
- Session coordinator → System audio capture: coordinates capture
- Microphone capture → Audio ring buffer: supplies samples
- System audio capture → Audio ring buffer: supplies samples
- Audio ring buffer → Voice activity detection: feeds audio
- Voice activity detection → Whisper transcription: gates speech
- Whisper transcription → Transcript assembly: emits transcripts
- Transcript assembly → Response evaluation: supplies segments
- Response evaluation → Conversation context: records turns
- Conversation context → Response evaluation: provides context
- Response evaluation → Prompt construction: requests prompt
- Prompt construction → Local language model: supplies prompt
- Local language model → Menu bar and UI: streams tokens
- Project context → Codebase analyzer: analyzes repositories
- Project context → Prompt construction: supplies project summary
- Project context → Local language model: saves project state
- Menu bar and UI → Interactive screen capture: starts capture
- Interactive screen capture → Vision processing: provides image
- Vision processing → Local language model: supplies visual input
- Session coordinator → Permission gateway: checks access
- Model management → Whisper transcription: validates model
- Model management → Local language model: validates model
- Model downloads → Meeting user: downloads models