How rufulosium/desktop-assistent works
Desktop Assistant is a voice-first macOS assistant with a permission-gated agent path and a notch-native interface. The main workflow begins with wake-word activation, captures speech, transcribes and routes it to an LLM or agent, then reflects activity in the UI. The graph also preserves documented audio effects, notifications, and provider choices. Some relationships come from README statements and the SOURCE INDEX; unsampled implementation details are not inferred, and the TTS implementation is not present in the supplied tree.
Main entry points: pipeline.py Assistant Pipeline · app.py App Lifecycle · capture.py Audio Capture · island_view.py Notch Island UI
Components 18
Application Runtime
- App Lifecycle
src/assistant/app.py - Configuration
src/assistant/config.py - Assistant Pipeline
src/assistant/pipeline.py
Audio and Speech
- Audio Capture
src/assistant/audio/capture.py - Audio Ring Buffer
src/assistant/audio/ringbuffer.py - Native Ring Buffer
native/ringbuffer/ringbuffer.h - Voice Activity Detection
src/assistant/audio/vad.py - Wake Detection
src/assistant/wake/sherpa_engine.py - Speech Recognition
src/assistant/stt/__init__.py
Language and Tasks
- Request Router
src/assistant/router/classifier.py - Local LLM
src/assistant/llm/local.py - Remote LLM
src/assistant/llm/remote.py - Agent Dispatch
src/assistant/agent/dispatch.py
Mac Experience
- Notch Island UI
src/assistant/ui/island_view.py - Island Window
src/assistant/ui/island_window.py - Sound Effects
src/assistant/audio/sounds.py - Notifications
src/assistant/notify.py
Other components
- User
Connections 21
- User → App Lifecycle: starts assistant
README.md - App Lifecycle → Configuration: uses settings
src/assistant/app.py - App Lifecycle → Assistant Pipeline: creates and initializes
src/assistant/app.py - App Lifecycle → Island Window: initializes UI
src/assistant/app.py - Assistant Pipeline → Wake Detection: runs wake detection
src/assistant/pipeline.py - Assistant Pipeline → Audio Capture: starts capture
src/assistant/pipeline.py - Audio Capture → Audio Ring Buffer: uses buffer
src/assistant/audio/capture.py - Audio Capture → Voice Activity Detection: uses VAD
src/assistant/audio/capture.py - Assistant Pipeline → Speech Recognition: transcribes audio
src/assistant/pipeline.py - Assistant Pipeline → Request Router: routes request
src/assistant/pipeline.py - Assistant Pipeline → Local LLM: requests completion
src/assistant/pipeline.py - Assistant Pipeline → Remote LLM: requests completion
src/assistant/pipeline.py - Assistant Pipeline → Agent Dispatch: dispatches tasks
src/assistant/pipeline.py - Assistant Pipeline → Sound Effects: plays effects
src/assistant/pipeline.py - Assistant Pipeline → Notifications: sends notifications
src/assistant/pipeline.py - Assistant Pipeline → Notch Island UI: updates UI state
- Island Window → Notch Island UI: hosts view
src/assistant/ui/island_window.py - User → Notch Island UI: drops files
README.md - Assistant Pipeline → Configuration: checks permissions
src/assistant/pipeline.py - Local LLM → User: uses local model service
src/assistant/llm/local.py - Remote LLM → User: uses provider API
README.md