How yliust/tactile works
Tactile is an accessibility-first operating layer for agents, delivered as platform skills and a macOS MCP tool server. The principal workflow is to inspect an application’s accessibility semantics, act through those semantics where possible, fall back to OCR-grounded coordinates and then visual operation when needed, and verify the result. The macOS SDK provides traversal, app opening, and input primitives; the MCP server exposes them to agents. README-described workflow stages are included, but unsampled wiring is omitted.
Main entry points: MacosUseSDK macOS SDK · main.swift MCP Server · SKILL.md macOS Skill · workflows macOS Workflows
Components 20
Agent Guidance
- macOS Skill
skills/tactile-macos/SKILL.md - App Guides
skills/tactile-macos/references/app-guides - macOS Workflows
skills/tactile-macos/scripts/workflows - Windows Skill
skills/tactile-windows/SKILL.md - Windows Guides
skills/tactile-windows/references/app-guides - Windows Workflow
skills/tactile-windows/scripts/run_workflow.py
macOS Runtime
- macOS SDK
skills/tactile-macos/scripts/MacosUseSDK/Sources/MacosUseSDK - AX Traversal
skills/tactile-macos/scripts/MacosUseSDK/Sources/MacosUseSDK/AccessibilityTraversal.swift - AX Actions
skills/tactile-macos/scripts/MacosUseSDK/Sources/MacosUseSDK/AccessibilityActions.swift - Input Control
skills/tactile-macos/scripts/MacosUseSDK/Sources/MacosUseSDK/InputController.swift - App Opening
skills/tactile-macos/scripts/MacosUseSDK/Sources/MacosUseSDK/AppOpener.swift - MCP Server
mcps/tactile-macos-mcp/Sources/MCPServer/main.swift - Screenshot Helper
mcps/tactile-macos-mcp/Sources/ScreenshotHelper/main.swift
Windows Runtime
- Windows Interface
skills/tactile-windows/scripts/windows_interface.py - macOS Interface
skills/tactile-macos/scripts/macos_interface.py
External Interfaces
- Target Application
- MCP Client
- LLM Service
Other components
- Agent
- Verified Result
Connections 20
- Agent → macOS Skill: uses guidance
- Agent → Windows Skill: uses guidance
- macOS Skill → App Guides: consults
- Windows Skill → Windows Guides: consults
- macOS Skill → macOS Workflows: guides tasks
- Windows Skill → Windows Workflow: guides tasks
- Agent → MCP Client: requests tools
- MCP Client → MCP Server: dispatches tools
- MCP Server → macOS SDK: uses runtime
- macOS SDK → AX Traversal: traverses UI
- macOS SDK → AX Actions: invokes AX
- macOS SDK → Input Control: simulates input
- macOS SDK → App Opening: opens apps
- MCP Server → Screenshot Helper: runs helper
- macOS Workflows → macOS Interface: runs interface
- Windows Workflow → Windows Interface: runs interface
- macOS SDK → Target Application: reads and controls
- MCP Server → Target Application: operates app
- Agent → Verified Result: verifies outcome
- macOS Workflows → LLM Service: requests reasoning