How kurojs/wayland-mcp works
Wayland MCP Server exposes desktop automation and screen-analysis tools to an MCP client. The principal workflow is a tool request routed by the server to screen capture or pointer/keyboard control; capability probing selects suitable backends, while optional VLM providers analyze or compare captured images. Action chains compose multiple automation operations. The README documents these capabilities, but several provider and backend implementations were not sampled, so those branches are represented at their supported directory paths; uncertain internal calls are omitted.
Main entry points: server_mcp.py Tool server · detect.py Backend selection · backends Input backends · screen_utils.py Screen utilities
Components 20
MCP interface
- Tool server
wayland_mcp/server_mcp.py
Desktop workflows
- Action chains
wayland_mcp/chain_processor.py - Screen utilities
wayland_mcp/screen_utils.py - Screen capture
wayland_mcp/app.py - Ruler overlays
wayland_mcp/add_rulers.py - Image analysis
wayland_mcp/app.py - Pointer control
wayland_mcp/mouse_utils.py - Keyboard control
wayland_mcp/keyboard_utils.py
Backend selection
- Capability probes
wayland_mcp/backends/base.py - Backend selection
wayland_mcp/backends/detect.py
System integrations
- Capture backends
wayland_mcp/backends - Input backends
wayland_mcp/backends - Portal transport
wayland_mcp/backends/portal.py - Key translation
wayland_mcp/backends/keycodes.py - Evemu keymap
wayland_mcp/keymap.py - Session recovery
wayland_mcp/session_env.py - Restore token
wayland_mcp/backends/input_portal.py
Other components
- MCP client
- Vision API
- Wayland desktop
Connections 24
- MCP client → Tool server: sends tools
README.md - Tool server → Action chains: dispatches chains
wayland_mcp/server_mcp.py - Tool server → Screen utilities: calls analysis
wayland_mcp/server_mcp.py - Tool server → Pointer control: routes pointer tools
wayland_mcp/server_mcp.py - Tool server → Keyboard control: routes keyboard tools
wayland_mcp/server_mcp.py - Tool server → Session recovery: recovers variables
wayland_mcp/server_mcp.py - Action chains → Pointer control: runs pointer actions
README.md - Action chains → Keyboard control: runs keyboard actions
README.md - Screen utilities → Screen capture: captures images
wayland_mcp/screen_utils.py - Screen utilities → Ruler overlays: adds overlays
wayland_mcp/screen_utils.py - Screen utilities → Image analysis: requests analysis
wayland_mcp/screen_utils.py - Image analysis → Vision API: sends images
wayland_mcp/app.py - Screen capture → Backend selection: selects backend
wayland_mcp/app.py - Backend selection → Capability probes: probes system
wayland_mcp/backends/detect.py - Backend selection → Capture backends: selects capture
wayland_mcp/backends/detect.py - Backend selection → Input backends: selects input
wayland_mcp/backends/detect.py - Capture backends → Wayland desktop: captures screen
README.md - Pointer control → Backend selection: selects backend
wayland_mcp/mouse_utils.py - Keyboard control → Backend selection: selects backend
wayland_mcp/keyboard_utils.py - Input backends → Wayland desktop: injects input
README.md - Input backends → Portal transport: uses portal calls
wayland_mcp/backends/input_portal.py - Input backends → Key translation: translates keys
wayland_mcp/backends/input_portal.py - Input backends → Evemu keymap: maps evemu keys
wayland_mcp/backends/input_evemu.py - Portal transport → Restore token: persists restore token
wayland_mcp/backends/input_portal.py