How zisshh/computer-use works
Alfred is a macOS voice-control assistant: it captures speech, transcribes it locally, interprets commands, and executes actions on the Mac or in supported apps. The main path runs through the session handler, which can use fast local/page-aware handling or ask Jev to select a typed action plan before execution. Supporting branches cover chat and YouTube-specific commands, audio gating and echo cancellation, screen context, and spoken/visual feedback. The README documents capabilities beyond the sampled excerpts; edges for those features cite it rather than implying unsampled implementation details were inspected.
Main entry points: main.py Session handler · audio.py Audio processing · brain.py Intent planner · actions.py macOS actions
Components 28
Voice input
- Session handler
alfred_computer_use/main.py - Audio capture
alfred_computer_use/recorder.py - Audio processing
alfred_computer_use/audio.py - Echo cancellation
alfred_computer_use/aec.py - Speech gate
alfred_computer_use/vad.py - Speech recognition
alfred_computer_use/stt.py - Apple recognition
alfred_computer_use/apple_stt.py - Streaming listener
alfred_computer_use/streaming.py
Command understanding
- Local command handling
alfred_computer_use/local.py - Intent planner
alfred_computer_use/brain.py - Command routing
alfred_computer_use/routing.py - Entity catalog
alfred_computer_use/catalog.py - Chat intent
alfred_computer_use/chat_intent.py
Action execution
- Action executor
alfred_computer_use/main.py - macOS actions
alfred_computer_use/actions.py - Apple Events
alfred_computer_use/osa.py
Context and feedback
- Screen context
alfred_computer_use/context.py - App focus
alfred_computer_use/focus.py - Status overlay
alfred_computer_use/overlay.py - Spoken replies
alfred_computer_use/tts.py - Runtime configuration
alfred_computer_use/config.py
App integrations
- Messages and WhatsApp
alfred_computer_use/chat.py - Discord controls
alfred_computer_use/discord.py - YouTube controls
alfred_computer_use/youtube.py - Page interaction
alfred_computer_use/page.py
Other components
- User
- Jev decision service
- Spotify service
Connections 27
- User → Session handler: speaks to
README.md - Session handler → Audio capture: captures audio
alfred_computer_use/main.py - Audio capture → Audio processing: provides audio
- Audio processing → Echo cancellation: cancels echo
alfred_computer_use/audio.py - Audio processing → Speech gate: gates speech
- Session handler → Streaming listener: starts listener
alfred_computer_use/main.py - Streaming listener → Speech recognition: transcribes audio
alfred_computer_use/streaming.py - Speech recognition → Apple recognition: selects engine
alfred_computer_use/stt.py - Session handler → Local command handling: checks locally
alfred_computer_use/main.py - Session handler → Intent planner: plans command
alfred_computer_use/main.py - Intent planner → Jev decision service: requests selection
README.md - Intent planner → Entity catalog: selects entities
alfred_computer_use/brain.py - Session handler → Command routing: resolves target
alfred_computer_use/main.py - Session handler → Action executor: dispatches plan
alfred_computer_use/main.py - Action executor → macOS actions: executes actions
alfred_computer_use/main.py - macOS actions → Apple Events: sends Apple events
alfred_computer_use/actions.py - Action executor → Screen context: reads context
alfred_computer_use/main.py - Action executor → App focus: checks focus
alfred_computer_use/main.py - Action executor → Messages and WhatsApp: handles chat
alfred_computer_use/main.py - Action executor → Chat intent: parses chat request
- Action executor → Discord controls: controls Discord
alfred_computer_use/main.py - Action executor → YouTube controls: handles YouTube
- Action executor → Page interaction: acts on page
- macOS actions → Spotify service: plays selected media
README.md - Session handler → Status overlay: updates status
alfred_computer_use/main.py - Session handler → Spoken replies: speaks reply
README.md - Session handler → Runtime configuration: loads settings
alfred_computer_use/main.py