# yliust/tactile architecture

> Architecture diagram of the GitHub repository yliust/tactile, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-09-27). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/yliust/tactile
- Repository: https://github.com/yliust/tactile

## Overview

Tactile is an accessibility-first operating layer for agents, delivered as platform skills and a macOS MCP tool server. The principal workflow is to inspect an application’s accessibility semantics, act through those semantics where possible, fall back to OCR-grounded coordinates and then visual operation when needed, and verify the result. The macOS SDK provides traversal, app opening, and input primitives; the MCP server exposes them to agents. README-described workflow stages are included, but unsampled wiring is omitted.

Main entry points: [`skills/tactile-macos/scripts/MacosUseSDK/Sources/MacosUseSDK`](https://github.com/yliust/tactile/tree/main/skills/tactile-macos/scripts/MacosUseSDK/Sources/MacosUseSDK) (macOS SDK), [`mcps/tactile-macos-mcp/Sources/MCPServer/main.swift`](https://github.com/yliust/tactile/blob/main/mcps/tactile-macos-mcp/Sources/MCPServer/main.swift) (MCP Server), [`skills/tactile-macos/SKILL.md`](https://github.com/yliust/tactile/blob/main/skills/tactile-macos/SKILL.md) (macOS Skill), [`skills/tactile-macos/scripts/workflows`](https://github.com/yliust/tactile/tree/main/skills/tactile-macos/scripts/workflows) (macOS Workflows)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_guidance["Agent Guidance"]
  node_macskill["macOS Skill<br/>[SKILL.md]"]
  node_appguides["App Guides"]
  node_macworkflow["macOS Workflows"]
  node_winskill["Windows Skill<br/>[SKILL.md]"]
  node_winguides["Windows Guides"]
  node_winworkflow["Windows Workflow<br/>[run_workflow.py]"]
end

subgraph group_macos["macOS Runtime"]
  node_macsdk["macOS SDK"]
  node_traversal["AX Traversal"]
  node_axactions["AX Actions"]
  node_input["Input Control"]
  node_appopen["App Opening<br/>[AppOpener.swift]"]
  node_mcp["MCP Server<br/>[main.swift]"]
  node_screenshot["Screenshot Helper<br/>[main.swift]"]
end

subgraph group_windows["Windows Runtime"]
  node_wininterface["Windows Interface"]
  node_macinterface["macOS Interface<br/>[macos_interface.py]"]
end

subgraph group_integrations["External Interfaces"]
  node_app["Target Application"]
  node_mcpclient(("MCP Client"))
  node_llm["LLM Service"]
end

node_agent(("Agent"))
node_result["Verified Result"]

node_agent -->|"uses guidance"| node_macskill
node_agent -->|"uses guidance"| node_winskill
node_macskill -->|"consults"| node_appguides
node_winskill -->|"consults"| node_winguides
node_macskill -->|"guides tasks"| node_macworkflow
node_winskill -->|"guides tasks"| node_winworkflow
node_agent -->|"requests tools"| node_mcpclient
node_mcpclient -->|"dispatches tools"| node_mcp
node_mcp -->|"uses runtime"| node_macsdk
node_macsdk -->|"traverses UI"| node_traversal
node_macsdk -->|"invokes AX"| node_axactions
node_macsdk -->|"simulates input"| node_input
node_macsdk -->|"opens apps"| node_appopen
node_mcp -->|"runs helper"| node_screenshot
node_macworkflow -->|"runs interface"| node_macinterface
node_winworkflow -->|"runs interface"| node_wininterface
node_macsdk -->|"reads and controls"| node_app
node_mcp -->|"operates app"| node_app
node_agent -->|"verifies outcome"| node_result
node_macworkflow -.->|"requests reasoning"| node_llm

click node_macskill "https://github.com/yliust/tactile/blob/main/skills/tactile-macos/SKILL.md"
click node_appguides "https://github.com/yliust/tactile/tree/main/skills/tactile-macos/references/app-guides"
click node_macworkflow "https://github.com/yliust/tactile/tree/main/skills/tactile-macos/scripts/workflows"
click node_winskill "https://github.com/yliust/tactile/blob/main/skills/tactile-windows/SKILL.md"
click node_winguides "https://github.com/yliust/tactile/tree/main/skills/tactile-windows/references/app-guides"
click node_winworkflow "https://github.com/yliust/tactile/blob/main/skills/tactile-windows/scripts/run_workflow.py"
click node_macsdk "https://github.com/yliust/tactile/tree/main/skills/tactile-macos/scripts/MacosUseSDK/Sources/MacosUseSDK"
click node_traversal "https://github.com/yliust/tactile/blob/main/skills/tactile-macos/scripts/MacosUseSDK/Sources/MacosUseSDK/AccessibilityTraversal.swift"
click node_axactions "https://github.com/yliust/tactile/blob/main/skills/tactile-macos/scripts/MacosUseSDK/Sources/MacosUseSDK/AccessibilityActions.swift"
click node_input "https://github.com/yliust/tactile/blob/main/skills/tactile-macos/scripts/MacosUseSDK/Sources/MacosUseSDK/InputController.swift"
click node_appopen "https://github.com/yliust/tactile/blob/main/skills/tactile-macos/scripts/MacosUseSDK/Sources/MacosUseSDK/AppOpener.swift"
click node_mcp "https://github.com/yliust/tactile/blob/main/mcps/tactile-macos-mcp/Sources/MCPServer/main.swift"
click node_screenshot "https://github.com/yliust/tactile/blob/main/mcps/tactile-macos-mcp/Sources/ScreenshotHelper/main.swift"
click node_wininterface "https://github.com/yliust/tactile/blob/main/skills/tactile-windows/scripts/windows_interface.py"
click node_macinterface "https://github.com/yliust/tactile/blob/main/skills/tactile-macos/scripts/macos_interface.py"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_macskill,node_appguides,node_macworkflow,node_winskill,node_winguides,node_winworkflow toneBlue
class node_macsdk,node_traversal,node_axactions,node_input,node_appopen,node_mcp,node_screenshot toneAmber
class node_wininterface,node_macinterface toneMint
class node_app,node_mcpclient,node_llm toneRose
class node_agent,node_result toneIndigo
```

## Components

### Agent Guidance

- [macOS Skill](https://github.com/yliust/tactile/blob/main/skills/tactile-macos/SKILL.md): `skills/tactile-macos/SKILL.md`
- [App Guides](https://github.com/yliust/tactile/tree/main/skills/tactile-macos/references/app-guides): `skills/tactile-macos/references/app-guides`
- [macOS Workflows](https://github.com/yliust/tactile/tree/main/skills/tactile-macos/scripts/workflows): `skills/tactile-macos/scripts/workflows`
- [Windows Skill](https://github.com/yliust/tactile/blob/main/skills/tactile-windows/SKILL.md): `skills/tactile-windows/SKILL.md`
- [Windows Guides](https://github.com/yliust/tactile/tree/main/skills/tactile-windows/references/app-guides): `skills/tactile-windows/references/app-guides`
- [Windows Workflow](https://github.com/yliust/tactile/blob/main/skills/tactile-windows/scripts/run_workflow.py): `skills/tactile-windows/scripts/run_workflow.py`

### macOS Runtime

- [macOS SDK](https://github.com/yliust/tactile/tree/main/skills/tactile-macos/scripts/MacosUseSDK/Sources/MacosUseSDK): `skills/tactile-macos/scripts/MacosUseSDK/Sources/MacosUseSDK`
- [AX Traversal](https://github.com/yliust/tactile/blob/main/skills/tactile-macos/scripts/MacosUseSDK/Sources/MacosUseSDK/AccessibilityTraversal.swift): `skills/tactile-macos/scripts/MacosUseSDK/Sources/MacosUseSDK/AccessibilityTraversal.swift`
- [AX Actions](https://github.com/yliust/tactile/blob/main/skills/tactile-macos/scripts/MacosUseSDK/Sources/MacosUseSDK/AccessibilityActions.swift): `skills/tactile-macos/scripts/MacosUseSDK/Sources/MacosUseSDK/AccessibilityActions.swift`
- [Input Control](https://github.com/yliust/tactile/blob/main/skills/tactile-macos/scripts/MacosUseSDK/Sources/MacosUseSDK/InputController.swift): `skills/tactile-macos/scripts/MacosUseSDK/Sources/MacosUseSDK/InputController.swift`
- [App Opening](https://github.com/yliust/tactile/blob/main/skills/tactile-macos/scripts/MacosUseSDK/Sources/MacosUseSDK/AppOpener.swift): `skills/tactile-macos/scripts/MacosUseSDK/Sources/MacosUseSDK/AppOpener.swift`
- [MCP Server](https://github.com/yliust/tactile/blob/main/mcps/tactile-macos-mcp/Sources/MCPServer/main.swift): `mcps/tactile-macos-mcp/Sources/MCPServer/main.swift`
- [Screenshot Helper](https://github.com/yliust/tactile/blob/main/mcps/tactile-macos-mcp/Sources/ScreenshotHelper/main.swift): `mcps/tactile-macos-mcp/Sources/ScreenshotHelper/main.swift`

### Windows Runtime

- [Windows Interface](https://github.com/yliust/tactile/blob/main/skills/tactile-windows/scripts/windows_interface.py): `skills/tactile-windows/scripts/windows_interface.py`
- [macOS Interface](https://github.com/yliust/tactile/blob/main/skills/tactile-macos/scripts/macos_interface.py): `skills/tactile-macos/scripts/macos_interface.py`

### External Interfaces

- **Target Application**
- **MCP Client**
- **LLM Service**

### Other components

- **Agent**
- **Verified Result**

## Connections

- Agent → macOS Skill: uses guidance
- Agent → Windows Skill: uses guidance
- macOS Skill → App Guides: consults
- Windows Skill → Windows Guides: consults
- macOS Skill → macOS Workflows: guides tasks
- Windows Skill → Windows Workflow: guides tasks
- Agent → MCP Client: requests tools
- MCP Client → MCP Server: dispatches tools
- MCP Server → macOS SDK: uses runtime
- macOS SDK → AX Traversal: traverses UI
- macOS SDK → AX Actions: invokes AX
- macOS SDK → Input Control: simulates input
- macOS SDK → App Opening: opens apps
- MCP Server → Screenshot Helper: runs helper
- macOS Workflows → macOS Interface: runs interface
- Windows Workflow → Windows Interface: runs interface
- macOS SDK → Target Application: reads and controls
- MCP Server → Target Application: operates app
- Agent → Verified Result: verifies outcome
- macOS Workflows → LLM Service: requests reasoning