# ghostwright/ghost-os architecture

> Architecture diagram of the GitHub repository ghostwright/ghost-os, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-10-01). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/ghostwright/ghost-os
- Repository: https://github.com/ghostwright/ghost-os

## Overview

Ghost OS is a macOS computer-use service for AI agents, exposed through MCP. The main workflow is an MCP client requesting perception or an action; tool dispatch routes to accessibility-based perception, optional visual grounding, and macOS input actions. Agents can also record manual workflows and replay parameterized JSON recipes. The graph preserves setup/diagnostics and screenshot support. Edges follow shown calls, source-index references, or README statements; unsampled implementations and sidecar details are not inferred beyond documented links.

Main entry points: [`Sources/GhostOS/MCP/MCPDispatch.swift`](https://github.com/ghostwright/ghost-os/blob/main/Sources/GhostOS/MCP/MCPDispatch.swift) (Tool dispatch), [`Sources/GhostOS/Recipes/RecipeEngine.swift`](https://github.com/ghostwright/ghost-os/blob/main/Sources/GhostOS/Recipes/RecipeEngine.swift) (Recipe execution), [`Sources/GhostOS/Perception/Perception.swift`](https://github.com/ghostwright/ghost-os/blob/main/Sources/GhostOS/Perception/Perception.swift) (AX perception), [`Sources/GhostOS/Actions/Actions.swift`](https://github.com/ghostwright/ghost-os/blob/main/Sources/GhostOS/Actions/Actions.swift) (Input actions)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_entry["Agent interface"]
  node_cli["Ghost CLI<br/>[main.swift]"]
  node_server["MCP server<br/>[MCPServer.swift]"]
  node_dispatch["Tool dispatch<br/>[MCPDispatch.swift]"]
end

subgraph group_observe["Perception"]
  node_perception["AX perception<br/>[Perception.swift]"]
  node_annotate["Screen annotation<br/>[Annotate.swift]"]
  node_capture["Screen capture"]
  node_cdp["CDP bridge<br/>[CDPBridge.swift]"]
  node_visionperception["Visual grounding"]
  node_visionbridge["Vision HTTP client<br/>[VisionBridge.swift]"]
  node_sidecar["Vision sidecar<br/>[server.py]"]
end

subgraph group_operate["Computer control"]
  node_actions["Input actions<br/>[Actions.swift]"]
  node_focus["Focus manager<br/>[FocusManager.swift]"]
  node_wait["Condition waits<br/>[WaitManager.swift]"]
end

subgraph group_workflow["Learning and recipes"]
  node_learningdispatch["Learning tools"]
  node_recorder["Action recorder"]
  node_eventhandlers["Event handlers"]
  node_switchdetector["App switch detector"]
  node_recipeengine["Recipe execution<br/>[RecipeEngine.swift]"]
  node_recipestore[("Recipe storage<br/>[RecipeStore.swift]")]
end

subgraph group_support["Local support"]
  node_setup["Setup wizard<br/>[SetupWizard.swift]"]
  node_doctor["Diagnostics<br/>[Doctor.swift]"]
end

node_agent(("MCP client"))

node_agent -->|"sends MCP"| node_server
node_cli -->|"starts"| node_server
node_cli -->|"runs"| node_setup
node_cli -->|"runs"| node_doctor
node_server -->|"routes tools"| node_dispatch
node_dispatch -->|"calls"| node_perception
node_dispatch -->|"calls"| node_annotate
node_dispatch -->|"calls"| node_actions
node_dispatch -->|"calls"| node_focus
node_dispatch -->|"calls"| node_wait
node_dispatch -->|"calls"| node_visionperception
node_dispatch -->|"calls"| node_learningdispatch
node_dispatch -->|"runs recipes"| node_recipeengine
node_dispatch -->|"manages recipes"| node_recipestore
node_perception -->|"captures screen"| node_capture
node_perception -->|"falls back"| node_cdp
node_perception -->|"falls back"| node_visionperception
node_annotate -->|"uses context"| node_perception
node_annotate -->|"captures screen"| node_capture
node_visionperception -->|"requests grounding"| node_visionbridge
node_visionperception -->|"captures screen"| node_capture
node_visionbridge -->|"HTTP inference"| node_sidecar
node_actions -->|"focuses app"| node_focus
node_actions -->|"locates elements"| node_perception
node_actions -->|"uses grounding"| node_visionperception
node_learningdispatch -->|"records actions"| node_recorder
node_recorder -->|"handles events"| node_eventhandlers
node_recorder -->|"detects switches"| node_switchdetector
node_recipeengine -->|"executes steps"| node_actions
node_recipeengine -->|"checks context"| node_perception
node_recipeengine -->|"manages focus"| node_focus
node_recipeengine -->|"waits for condition"| node_wait
node_recipeengine -.->|"loads recipe"| node_recipestore
node_agent -->|"learns workflows"| node_learningdispatch
node_agent -->|"runs recipes"| node_recipeengine
node_setup -->|"installs recipes"| node_recipestore
node_doctor -->|"checks permission"| node_capture
node_doctor -->|"checks recipes"| node_recipestore
node_doctor -->|"checks vision"| node_visionbridge

click node_cli "https://github.com/ghostwright/ghost-os/blob/main/Sources/ghost/main.swift"
click node_server "https://github.com/ghostwright/ghost-os/blob/main/Sources/GhostOS/MCP/MCPServer.swift"
click node_dispatch "https://github.com/ghostwright/ghost-os/blob/main/Sources/GhostOS/MCP/MCPDispatch.swift"
click node_perception "https://github.com/ghostwright/ghost-os/blob/main/Sources/GhostOS/Perception/Perception.swift"
click node_annotate "https://github.com/ghostwright/ghost-os/blob/main/Sources/GhostOS/Perception/Annotate.swift"
click node_capture "https://github.com/ghostwright/ghost-os/blob/main/Sources/GhostOS/Screenshot/ScreenCapture.swift"
click node_cdp "https://github.com/ghostwright/ghost-os/blob/main/Sources/GhostOS/Vision/CDPBridge.swift"
click node_visionperception "https://github.com/ghostwright/ghost-os/blob/main/Sources/GhostOS/Vision/VisionPerception.swift"
click node_visionbridge "https://github.com/ghostwright/ghost-os/blob/main/Sources/GhostOS/Vision/VisionBridge.swift"
click node_sidecar "https://github.com/ghostwright/ghost-os/blob/main/vision-sidecar/server.py"
click node_actions "https://github.com/ghostwright/ghost-os/blob/main/Sources/GhostOS/Actions/Actions.swift"
click node_focus "https://github.com/ghostwright/ghost-os/blob/main/Sources/GhostOS/Actions/FocusManager.swift"
click node_wait "https://github.com/ghostwright/ghost-os/blob/main/Sources/GhostOS/MCP/WaitManager.swift"
click node_learningdispatch "https://github.com/ghostwright/ghost-os/blob/main/Sources/GhostOS/Learning/LearningDispatch.swift"
click node_recorder "https://github.com/ghostwright/ghost-os/blob/main/Sources/GhostOS/Learning/LearningRecorder.swift"
click node_eventhandlers "https://github.com/ghostwright/ghost-os/blob/main/Sources/GhostOS/Learning/EventHandlers.swift"
click node_switchdetector "https://github.com/ghostwright/ghost-os/blob/main/Sources/GhostOS/Learning/AppSwitchDetector.swift"
click node_recipeengine "https://github.com/ghostwright/ghost-os/blob/main/Sources/GhostOS/Recipes/RecipeEngine.swift"
click node_recipestore "https://github.com/ghostwright/ghost-os/blob/main/Sources/GhostOS/Recipes/RecipeStore.swift"
click node_setup "https://github.com/ghostwright/ghost-os/blob/main/Sources/ghost/SetupWizard.swift"
click node_doctor "https://github.com/ghostwright/ghost-os/blob/main/Sources/ghost/Doctor.swift"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_cli,node_server,node_dispatch,node_agent toneBlue
class node_perception,node_annotate,node_capture,node_cdp,node_visionperception,node_visionbridge,node_sidecar toneAmber
class node_actions,node_focus,node_wait toneMint
class node_learningdispatch,node_recorder,node_eventhandlers,node_switchdetector,node_recipeengine,node_recipestore toneRose
class node_setup,node_doctor toneIndigo
```

## Components

### Agent interface

- [Ghost CLI](https://github.com/ghostwright/ghost-os/blob/main/Sources/ghost/main.swift): `Sources/ghost/main.swift`
- [MCP server](https://github.com/ghostwright/ghost-os/blob/main/Sources/GhostOS/MCP/MCPServer.swift): `Sources/GhostOS/MCP/MCPServer.swift`
- [Tool dispatch](https://github.com/ghostwright/ghost-os/blob/main/Sources/GhostOS/MCP/MCPDispatch.swift): `Sources/GhostOS/MCP/MCPDispatch.swift`

### Perception

- [AX perception](https://github.com/ghostwright/ghost-os/blob/main/Sources/GhostOS/Perception/Perception.swift): `Sources/GhostOS/Perception/Perception.swift`
- [Screen annotation](https://github.com/ghostwright/ghost-os/blob/main/Sources/GhostOS/Perception/Annotate.swift): `Sources/GhostOS/Perception/Annotate.swift`
- [Screen capture](https://github.com/ghostwright/ghost-os/blob/main/Sources/GhostOS/Screenshot/ScreenCapture.swift): `Sources/GhostOS/Screenshot/ScreenCapture.swift`
- [CDP bridge](https://github.com/ghostwright/ghost-os/blob/main/Sources/GhostOS/Vision/CDPBridge.swift): `Sources/GhostOS/Vision/CDPBridge.swift`
- [Visual grounding](https://github.com/ghostwright/ghost-os/blob/main/Sources/GhostOS/Vision/VisionPerception.swift): `Sources/GhostOS/Vision/VisionPerception.swift`
- [Vision HTTP client](https://github.com/ghostwright/ghost-os/blob/main/Sources/GhostOS/Vision/VisionBridge.swift): `Sources/GhostOS/Vision/VisionBridge.swift`
- [Vision sidecar](https://github.com/ghostwright/ghost-os/blob/main/vision-sidecar/server.py): `vision-sidecar/server.py`

### Computer control

- [Input actions](https://github.com/ghostwright/ghost-os/blob/main/Sources/GhostOS/Actions/Actions.swift): `Sources/GhostOS/Actions/Actions.swift`
- [Focus manager](https://github.com/ghostwright/ghost-os/blob/main/Sources/GhostOS/Actions/FocusManager.swift): `Sources/GhostOS/Actions/FocusManager.swift`
- [Condition waits](https://github.com/ghostwright/ghost-os/blob/main/Sources/GhostOS/MCP/WaitManager.swift): `Sources/GhostOS/MCP/WaitManager.swift`

### Learning and recipes

- [Learning tools](https://github.com/ghostwright/ghost-os/blob/main/Sources/GhostOS/Learning/LearningDispatch.swift): `Sources/GhostOS/Learning/LearningDispatch.swift`
- [Action recorder](https://github.com/ghostwright/ghost-os/blob/main/Sources/GhostOS/Learning/LearningRecorder.swift): `Sources/GhostOS/Learning/LearningRecorder.swift`
- [Event handlers](https://github.com/ghostwright/ghost-os/blob/main/Sources/GhostOS/Learning/EventHandlers.swift): `Sources/GhostOS/Learning/EventHandlers.swift`
- [App switch detector](https://github.com/ghostwright/ghost-os/blob/main/Sources/GhostOS/Learning/AppSwitchDetector.swift): `Sources/GhostOS/Learning/AppSwitchDetector.swift`
- [Recipe execution](https://github.com/ghostwright/ghost-os/blob/main/Sources/GhostOS/Recipes/RecipeEngine.swift): `Sources/GhostOS/Recipes/RecipeEngine.swift`
- [Recipe storage](https://github.com/ghostwright/ghost-os/blob/main/Sources/GhostOS/Recipes/RecipeStore.swift): `Sources/GhostOS/Recipes/RecipeStore.swift`

### Local support

- [Setup wizard](https://github.com/ghostwright/ghost-os/blob/main/Sources/ghost/SetupWizard.swift): `Sources/ghost/SetupWizard.swift`
- [Diagnostics](https://github.com/ghostwright/ghost-os/blob/main/Sources/ghost/Doctor.swift): `Sources/ghost/Doctor.swift`

### Other components

- **MCP client**

## Connections

- MCP client → MCP server: sends MCP (evidence: `README.md`)
- Ghost CLI → MCP server: starts (evidence: `Sources/ghost/main.swift`)
- Ghost CLI → Setup wizard: runs (evidence: `Sources/ghost/main.swift`)
- Ghost CLI → Diagnostics: runs (evidence: `Sources/ghost/main.swift`)
- MCP server → Tool dispatch: routes tools (evidence: `Sources/GhostOS/MCP/MCPServer.swift`)
- Tool dispatch → AX perception: calls (evidence: `Sources/GhostOS/MCP/MCPDispatch.swift`)
- Tool dispatch → Screen annotation: calls (evidence: `Sources/GhostOS/MCP/MCPDispatch.swift`)
- Tool dispatch → Input actions: calls (evidence: `Sources/GhostOS/MCP/MCPDispatch.swift`)
- Tool dispatch → Focus manager: calls (evidence: `Sources/GhostOS/MCP/MCPDispatch.swift`)
- Tool dispatch → Condition waits: calls (evidence: `Sources/GhostOS/MCP/MCPDispatch.swift`)
- Tool dispatch → Visual grounding: calls (evidence: `Sources/GhostOS/MCP/MCPDispatch.swift`)
- Tool dispatch → Learning tools: calls (evidence: `Sources/GhostOS/MCP/MCPDispatch.swift`)
- Tool dispatch → Recipe execution: runs recipes (evidence: `Sources/GhostOS/MCP/MCPDispatch.swift`)
- Tool dispatch → Recipe storage: manages recipes (evidence: `Sources/GhostOS/MCP/MCPDispatch.swift`)
- AX perception → Screen capture: captures screen (evidence: `Sources/GhostOS/Perception/Perception.swift`)
- AX perception → CDP bridge: falls back (evidence: `Sources/GhostOS/Perception/Perception.swift`)
- AX perception → Visual grounding: falls back (evidence: `Sources/GhostOS/Perception/Perception.swift`)
- Screen annotation → AX perception: uses context (evidence: `Sources/GhostOS/Perception/Annotate.swift`)
- Screen annotation → Screen capture: captures screen (evidence: `Sources/GhostOS/Perception/Annotate.swift`)
- Visual grounding → Vision HTTP client: requests grounding (evidence: `Sources/GhostOS/Vision/VisionPerception.swift`)
- Visual grounding → Screen capture: captures screen (evidence: `Sources/GhostOS/Vision/VisionPerception.swift`)
- Vision HTTP client → Vision sidecar: HTTP inference (evidence: `Sources/GhostOS/Vision/VisionBridge.swift`)
- Input actions → Focus manager: focuses app (evidence: `Sources/GhostOS/Actions/Actions.swift`)
- Input actions → AX perception: locates elements (evidence: `Sources/GhostOS/Actions/Actions.swift`)
- Input actions → Visual grounding: uses grounding (evidence: `Sources/GhostOS/Actions/Actions.swift`)
- Learning tools → Action recorder: records actions (evidence: `Sources/GhostOS/Learning/LearningDispatch.swift`)
- Action recorder → Event handlers: handles events (evidence: `Sources/GhostOS/Learning/LearningRecorder.swift`)
- Action recorder → App switch detector: detects switches (evidence: `Sources/GhostOS/Learning/LearningRecorder.swift`)
- Recipe execution → Input actions: executes steps (evidence: `Sources/GhostOS/Recipes/RecipeEngine.swift`)
- Recipe execution → AX perception: checks context (evidence: `Sources/GhostOS/Recipes/RecipeEngine.swift`)
- Recipe execution → Focus manager: manages focus (evidence: `Sources/GhostOS/Recipes/RecipeEngine.swift`)
- Recipe execution → Condition waits: waits for condition (evidence: `Sources/GhostOS/Recipes/RecipeEngine.swift`)
- Recipe execution → Recipe storage: loads recipe
- MCP client → Learning tools: learns workflows (evidence: `README.md`)
- MCP client → Recipe execution: runs recipes (evidence: `README.md`)
- Setup wizard → Recipe storage: installs recipes (evidence: `README.md`)
- Diagnostics → Screen capture: checks permission (evidence: `Sources/ghost/Doctor.swift`)
- Diagnostics → Recipe storage: checks recipes (evidence: `Sources/ghost/Doctor.swift`)
- Diagnostics → Vision HTTP client: checks vision (evidence: `Sources/ghost/Doctor.swift`)