# ifuryst/open-computer-use architecture

> Architecture diagram of the GitHub repository ifuryst/open-computer-use, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-11-12). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/ifuryst/open-computer-use
- Repository: https://github.com/ifuryst/open-computer-use

## Overview

open-computer-use is a cross-platform computer-use service exposed through MCP. An MCP client requests tools; the runtime discovers applications, reads accessibility state, and performs computer actions. macOS additionally offers permission onboarding and a Locked Use mode, whose broker coordinates a guarded temporary-unlock lifecycle. The graph preserves documented Linux and Windows runtimes and major Locked Use stages, but their internal wiring is not sampled here; dashed links mark those documented or unverified boundaries.

Main entry points: [`experiments/LockedUse/Sources/Broker/BrokerMain.swift`](https://github.com/ifuryst/open-computer-use/blob/main/experiments/LockedUse/Sources/Broker/BrokerMain.swift) (Privileged broker), [`packages/OpenComputerUseKit/Sources/OpenComputerUseKit/ComputerUseService.swift`](https://github.com/ifuryst/open-computer-use/blob/main/packages/OpenComputerUseKit/Sources/OpenComputerUseKit/ComputerUseService.swift) (Computer-use service), [`apps/OpenComputerUse/Sources/OpenComputerUse/MacOSAppAgentProxy.swift`](https://github.com/ifuryst/open-computer-use/blob/main/apps/OpenComputerUse/Sources/OpenComputerUse/MacOSAppAgentProxy.swift) (macOS agent proxy), [`experiments/LockedUse/Sources/Installer/InstallerMain.swift`](https://github.com/ifuryst/open-computer-use/blob/main/experiments/LockedUse/Sources/Installer/InstallerMain.swift) (Locked Use installer)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_entry["Client and Runtime"]
  node_mcp["MCP server<br/>[MCPServer.swift]"]
  node_cli["CLI commands"]
  node_macproxy["macOS agent proxy"]
end

subgraph group_computer["Computer Use"]
  node_service["Computer-use service"]
  node_discovery["App discovery<br/>[AppDiscovery.swift]"]
  node_fixture["Fixture bridge"]
  node_cursor["Software cursor"]
  node_selection["Text selection"]
  node_onboarding["Permission onboarding"]
end

subgraph group_locked["Locked Use"]
  node_broker["Privileged broker<br/>[BrokerMain.swift]"]
  node_coordinator["Lease coordinator"]
  node_state["Lease state machine"]
  node_approvals["Client approvals"]
  node_rules["Authorization rules"]
  node_recovery[("Recovery fence")]
  node_guardian["Display guardian"]
  node_installer["Locked Use installer"]
end

subgraph group_platforms["Platform Runtimes"]
  node_linux["Linux runtime<br/>[main.go]"]
  node_linuxpy["Linux platform layer<br/>[runtime.py]"]
  node_windows["Windows runtime<br/>[main.go]"]
end

node_agent(("AI / MCP client"))

node_agent -.->|"invokes tools"| node_mcp
node_mcp -.->|"dispatches tools"| node_service
node_cli -.->|"runs actions"| node_service
node_macproxy -->|"uses service"| node_service
node_service -->|"discovers apps"| node_discovery
node_service -->|"posts fixture commands"| node_fixture
node_service -->|"updates overlay"| node_cursor
node_service -->|"resolves selection"| node_selection
node_macproxy -->|"presents onboarding"| node_onboarding
node_agent -.->|"uses MCP service"| node_linux
node_agent -.->|"uses MCP service"| node_windows
node_linux -.->|"uses platform layer"| node_linuxpy
node_macproxy -.->|"requests Locked Use"| node_broker
node_broker -->|"loads approvals"| node_approvals
node_broker -->|"coordinates leases"| node_coordinator
node_coordinator -->|"drives transitions"| node_state
node_broker -->|"loads policy"| node_rules
node_broker -->|"persists recovery"| node_recovery
node_broker -.->|"coordinates protection"| node_guardian
node_installer -->|"configures policy"| node_rules
node_installer -->|"configures approvals"| node_approvals
node_installer -.->|"installs service"| node_broker

click node_mcp "https://github.com/ifuryst/open-computer-use/blob/main/packages/OpenComputerUseKit/Sources/OpenComputerUseKit/MCPServer.swift"
click node_cli "https://github.com/ifuryst/open-computer-use/blob/main/packages/OpenComputerUseKit/Sources/OpenComputerUseKit/OpenComputerUseCLI.swift"
click node_macproxy "https://github.com/ifuryst/open-computer-use/blob/main/apps/OpenComputerUse/Sources/OpenComputerUse/MacOSAppAgentProxy.swift"
click node_service "https://github.com/ifuryst/open-computer-use/blob/main/packages/OpenComputerUseKit/Sources/OpenComputerUseKit/ComputerUseService.swift"
click node_discovery "https://github.com/ifuryst/open-computer-use/blob/main/packages/OpenComputerUseKit/Sources/OpenComputerUseKit/AppDiscovery.swift"
click node_fixture "https://github.com/ifuryst/open-computer-use/blob/main/packages/OpenComputerUseKit/Sources/OpenComputerUseKit/FixtureBridge.swift"
click node_cursor "https://github.com/ifuryst/open-computer-use/blob/main/packages/OpenComputerUseKit/Sources/OpenComputerUseKit/SoftwareCursorOverlay.swift"
click node_selection "https://github.com/ifuryst/open-computer-use/blob/main/packages/OpenComputerUseKit/Sources/OpenComputerUseKit/TextSelectionResolver.swift"
click node_onboarding "https://github.com/ifuryst/open-computer-use/blob/main/apps/OpenComputerUse/Sources/OpenComputerUse/PermissionOnboardingApp.swift"
click node_linux "https://github.com/ifuryst/open-computer-use/blob/main/apps/OpenComputerUseLinux/main.go"
click node_linuxpy "https://github.com/ifuryst/open-computer-use/blob/main/apps/OpenComputerUseLinux/runtime.py"
click node_windows "https://github.com/ifuryst/open-computer-use/blob/main/apps/OpenComputerUseWindows/main.go"
click node_broker "https://github.com/ifuryst/open-computer-use/blob/main/experiments/LockedUse/Sources/Broker/BrokerMain.swift"
click node_coordinator "https://github.com/ifuryst/open-computer-use/blob/main/packages/OpenComputerUseKit/Sources/OpenComputerUseKit/LockedUseBrokerCoordinator.swift"
click node_state "https://github.com/ifuryst/open-computer-use/blob/main/packages/OpenComputerUseKit/Sources/OpenComputerUseKit/LockedUseStateMachine.swift"
click node_approvals "https://github.com/ifuryst/open-computer-use/blob/main/packages/OpenComputerUseKit/Sources/OpenComputerUseKit/LockedUseClientApprovals.swift"
click node_rules "https://github.com/ifuryst/open-computer-use/blob/main/packages/OpenComputerUseKit/Sources/OpenComputerUseKit/LockedUseAuthorizationRules.swift"
click node_recovery "https://github.com/ifuryst/open-computer-use/blob/main/packages/OpenComputerUseKit/Sources/OpenComputerUseKit/LockedUseRecoveryRecord.swift"
click node_guardian "https://github.com/ifuryst/open-computer-use/blob/main/experiments/LockedUse/Sources/Guardian/DisplayGuardian.swift"
click node_installer "https://github.com/ifuryst/open-computer-use/blob/main/experiments/LockedUse/Sources/Installer/InstallerMain.swift"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_mcp,node_cli,node_macproxy,node_agent toneBlue
class node_service,node_discovery,node_fixture,node_cursor,node_selection,node_onboarding toneAmber
class node_broker,node_coordinator,node_state,node_approvals,node_rules,node_recovery,node_guardian,node_installer toneMint
class node_linux,node_linuxpy,node_windows toneRose
```

## Components

### Client and Runtime

- [MCP server](https://github.com/ifuryst/open-computer-use/blob/main/packages/OpenComputerUseKit/Sources/OpenComputerUseKit/MCPServer.swift): `packages/OpenComputerUseKit/Sources/OpenComputerUseKit/MCPServer.swift`
- [CLI commands](https://github.com/ifuryst/open-computer-use/blob/main/packages/OpenComputerUseKit/Sources/OpenComputerUseKit/OpenComputerUseCLI.swift): `packages/OpenComputerUseKit/Sources/OpenComputerUseKit/OpenComputerUseCLI.swift`
- [macOS agent proxy](https://github.com/ifuryst/open-computer-use/blob/main/apps/OpenComputerUse/Sources/OpenComputerUse/MacOSAppAgentProxy.swift): `apps/OpenComputerUse/Sources/OpenComputerUse/MacOSAppAgentProxy.swift`

### Computer Use

- [Computer-use service](https://github.com/ifuryst/open-computer-use/blob/main/packages/OpenComputerUseKit/Sources/OpenComputerUseKit/ComputerUseService.swift): `packages/OpenComputerUseKit/Sources/OpenComputerUseKit/ComputerUseService.swift`
- [App discovery](https://github.com/ifuryst/open-computer-use/blob/main/packages/OpenComputerUseKit/Sources/OpenComputerUseKit/AppDiscovery.swift): `packages/OpenComputerUseKit/Sources/OpenComputerUseKit/AppDiscovery.swift`
- [Fixture bridge](https://github.com/ifuryst/open-computer-use/blob/main/packages/OpenComputerUseKit/Sources/OpenComputerUseKit/FixtureBridge.swift): `packages/OpenComputerUseKit/Sources/OpenComputerUseKit/FixtureBridge.swift`
- [Software cursor](https://github.com/ifuryst/open-computer-use/blob/main/packages/OpenComputerUseKit/Sources/OpenComputerUseKit/SoftwareCursorOverlay.swift): `packages/OpenComputerUseKit/Sources/OpenComputerUseKit/SoftwareCursorOverlay.swift`
- [Text selection](https://github.com/ifuryst/open-computer-use/blob/main/packages/OpenComputerUseKit/Sources/OpenComputerUseKit/TextSelectionResolver.swift): `packages/OpenComputerUseKit/Sources/OpenComputerUseKit/TextSelectionResolver.swift`
- [Permission onboarding](https://github.com/ifuryst/open-computer-use/blob/main/apps/OpenComputerUse/Sources/OpenComputerUse/PermissionOnboardingApp.swift): `apps/OpenComputerUse/Sources/OpenComputerUse/PermissionOnboardingApp.swift`

### Locked Use

- [Privileged broker](https://github.com/ifuryst/open-computer-use/blob/main/experiments/LockedUse/Sources/Broker/BrokerMain.swift): `experiments/LockedUse/Sources/Broker/BrokerMain.swift`
- [Lease coordinator](https://github.com/ifuryst/open-computer-use/blob/main/packages/OpenComputerUseKit/Sources/OpenComputerUseKit/LockedUseBrokerCoordinator.swift): `packages/OpenComputerUseKit/Sources/OpenComputerUseKit/LockedUseBrokerCoordinator.swift`
- [Lease state machine](https://github.com/ifuryst/open-computer-use/blob/main/packages/OpenComputerUseKit/Sources/OpenComputerUseKit/LockedUseStateMachine.swift): `packages/OpenComputerUseKit/Sources/OpenComputerUseKit/LockedUseStateMachine.swift`
- [Client approvals](https://github.com/ifuryst/open-computer-use/blob/main/packages/OpenComputerUseKit/Sources/OpenComputerUseKit/LockedUseClientApprovals.swift): `packages/OpenComputerUseKit/Sources/OpenComputerUseKit/LockedUseClientApprovals.swift`
- [Authorization rules](https://github.com/ifuryst/open-computer-use/blob/main/packages/OpenComputerUseKit/Sources/OpenComputerUseKit/LockedUseAuthorizationRules.swift): `packages/OpenComputerUseKit/Sources/OpenComputerUseKit/LockedUseAuthorizationRules.swift`
- [Recovery fence](https://github.com/ifuryst/open-computer-use/blob/main/packages/OpenComputerUseKit/Sources/OpenComputerUseKit/LockedUseRecoveryRecord.swift): `packages/OpenComputerUseKit/Sources/OpenComputerUseKit/LockedUseRecoveryRecord.swift`
- [Display guardian](https://github.com/ifuryst/open-computer-use/blob/main/experiments/LockedUse/Sources/Guardian/DisplayGuardian.swift): `experiments/LockedUse/Sources/Guardian/DisplayGuardian.swift`
- [Locked Use installer](https://github.com/ifuryst/open-computer-use/blob/main/experiments/LockedUse/Sources/Installer/InstallerMain.swift): `experiments/LockedUse/Sources/Installer/InstallerMain.swift`

### Platform Runtimes

- [Linux runtime](https://github.com/ifuryst/open-computer-use/blob/main/apps/OpenComputerUseLinux/main.go): `apps/OpenComputerUseLinux/main.go`
- [Linux platform layer](https://github.com/ifuryst/open-computer-use/blob/main/apps/OpenComputerUseLinux/runtime.py): `apps/OpenComputerUseLinux/runtime.py`
- [Windows runtime](https://github.com/ifuryst/open-computer-use/blob/main/apps/OpenComputerUseWindows/main.go): `apps/OpenComputerUseWindows/main.go`

### Other components

- **AI / MCP client**

## Connections

- AI / MCP client → MCP server: invokes tools (evidence: `README.md`)
- MCP server → Computer-use service: dispatches tools (evidence: `README.md`)
- CLI commands → Computer-use service: runs actions (evidence: `README.md`)
- macOS agent proxy → Computer-use service: uses service (evidence: `apps/OpenComputerUse/Sources/OpenComputerUse/MacOSAppAgentProxy.swift`)
- Computer-use service → App discovery: discovers apps (evidence: `packages/OpenComputerUseKit/Sources/OpenComputerUseKit/ComputerUseService.swift`)
- Computer-use service → Fixture bridge: posts fixture commands (evidence: `packages/OpenComputerUseKit/Sources/OpenComputerUseKit/ComputerUseService.swift`)
- Computer-use service → Software cursor: updates overlay (evidence: `packages/OpenComputerUseKit/Sources/OpenComputerUseKit/ComputerUseService.swift`)
- Computer-use service → Text selection: resolves selection (evidence: `packages/OpenComputerUseKit/Sources/OpenComputerUseKit/ComputerUseService.swift`)
- macOS agent proxy → Permission onboarding: presents onboarding (evidence: `apps/OpenComputerUse/Sources/OpenComputerUse/MacOSAppAgentProxy.swift`)
- AI / MCP client → Linux runtime: uses MCP service (evidence: `README.md`)
- AI / MCP client → Windows runtime: uses MCP service (evidence: `README.md`)
- Linux runtime → Linux platform layer: uses platform layer
- macOS agent proxy → Privileged broker: requests Locked Use (evidence: `README.md`)
- Privileged broker → Client approvals: loads approvals (evidence: `experiments/LockedUse/Sources/Broker/BrokerMain.swift`)
- Privileged broker → Lease coordinator: coordinates leases (evidence: `experiments/LockedUse/Sources/Broker/BrokerMain.swift`)
- Lease coordinator → Lease state machine: drives transitions (evidence: `packages/OpenComputerUseKit/Sources/OpenComputerUseKit/LockedUseBrokerCoordinator.swift`)
- Privileged broker → Authorization rules: loads policy (evidence: `experiments/LockedUse/Sources/Broker/BrokerMain.swift`)
- Privileged broker → Recovery fence: persists recovery (evidence: `experiments/LockedUse/Sources/Broker/BrokerMain.swift`)
- Privileged broker → Display guardian: coordinates protection
- Locked Use installer → Authorization rules: configures policy (evidence: `experiments/LockedUse/Sources/Installer/InstallerMain.swift`)
- Locked Use installer → Client approvals: configures approvals (evidence: `experiments/LockedUse/Sources/Installer/InstallerMain.swift`)
- Locked Use installer → Privileged broker: installs service