How ifuryst/open-computer-use works
open-computer-use is a cross-platform computer-use service exposed through MCP. An MCP client requests tools; the runtime discovers applications, reads accessibility state, and performs computer actions. macOS additionally offers permission onboarding and a Locked Use mode, whose broker coordinates a guarded temporary-unlock lifecycle. The graph preserves documented Linux and Windows runtimes and major Locked Use stages, but their internal wiring is not sampled here; dashed links mark those documented or unverified boundaries.
Main entry points: BrokerMain.swift Privileged broker · ComputerUseService.swift Computer-use service · MacOSAppAgentProxy.swift macOS agent proxy · InstallerMain.swift Locked Use installer
Components 21
Client and Runtime
- MCP server
packages/OpenComputerUseKit/Sources/OpenComputerUseKit/MCPServer.swift - CLI commands
packages/OpenComputerUseKit/Sources/OpenComputerUseKit/OpenComputerUseCLI.swift - macOS agent proxy
apps/OpenComputerUse/Sources/OpenComputerUse/MacOSAppAgentProxy.swift
Computer Use
- Computer-use service
packages/OpenComputerUseKit/Sources/OpenComputerUseKit/ComputerUseService.swift - App discovery
packages/OpenComputerUseKit/Sources/OpenComputerUseKit/AppDiscovery.swift - Fixture bridge
packages/OpenComputerUseKit/Sources/OpenComputerUseKit/FixtureBridge.swift - Software cursor
packages/OpenComputerUseKit/Sources/OpenComputerUseKit/SoftwareCursorOverlay.swift - Text selection
packages/OpenComputerUseKit/Sources/OpenComputerUseKit/TextSelectionResolver.swift - Permission onboarding
apps/OpenComputerUse/Sources/OpenComputerUse/PermissionOnboardingApp.swift
Locked Use
- Privileged broker
experiments/LockedUse/Sources/Broker/BrokerMain.swift - Lease coordinator
packages/OpenComputerUseKit/Sources/OpenComputerUseKit/LockedUseBrokerCoordinator.swift - Lease state machine
packages/OpenComputerUseKit/Sources/OpenComputerUseKit/LockedUseStateMachine.swift - Client approvals
packages/OpenComputerUseKit/Sources/OpenComputerUseKit/LockedUseClientApprovals.swift - Authorization rules
packages/OpenComputerUseKit/Sources/OpenComputerUseKit/LockedUseAuthorizationRules.swift - Recovery fence
packages/OpenComputerUseKit/Sources/OpenComputerUseKit/LockedUseRecoveryRecord.swift - Display guardian
experiments/LockedUse/Sources/Guardian/DisplayGuardian.swift - Locked Use installer
experiments/LockedUse/Sources/Installer/InstallerMain.swift
Platform Runtimes
- Linux runtime
apps/OpenComputerUseLinux/main.go - Linux platform layer
apps/OpenComputerUseLinux/runtime.py - Windows runtime
apps/OpenComputerUseWindows/main.go
Other components
- AI / MCP client
Connections 22
- AI / MCP client → MCP server: invokes tools
README.md - MCP server → Computer-use service: dispatches tools
README.md - CLI commands → Computer-use service: runs actions
README.md - macOS agent proxy → Computer-use service: uses service
apps/OpenComputerUse/Sources/OpenComputerUse/MacOSAppAgentProxy.swift - Computer-use service → App discovery: discovers apps
packages/OpenComputerUseKit/Sources/OpenComputerUseKit/ComputerUseService.swift - Computer-use service → Fixture bridge: posts fixture commands
packages/OpenComputerUseKit/Sources/OpenComputerUseKit/ComputerUseService.swift - Computer-use service → Software cursor: updates overlay
packages/OpenComputerUseKit/Sources/OpenComputerUseKit/ComputerUseService.swift - Computer-use service → Text selection: resolves selection
packages/OpenComputerUseKit/Sources/OpenComputerUseKit/ComputerUseService.swift - macOS agent proxy → Permission onboarding: presents onboarding
apps/OpenComputerUse/Sources/OpenComputerUse/MacOSAppAgentProxy.swift - AI / MCP client → Linux runtime: uses MCP service
README.md - AI / MCP client → Windows runtime: uses MCP service
README.md - Linux runtime → Linux platform layer: uses platform layer
- macOS agent proxy → Privileged broker: requests Locked Use
README.md - Privileged broker → Client approvals: loads approvals
experiments/LockedUse/Sources/Broker/BrokerMain.swift - Privileged broker → Lease coordinator: coordinates leases
experiments/LockedUse/Sources/Broker/BrokerMain.swift - Lease coordinator → Lease state machine: drives transitions
packages/OpenComputerUseKit/Sources/OpenComputerUseKit/LockedUseBrokerCoordinator.swift - Privileged broker → Authorization rules: loads policy
experiments/LockedUse/Sources/Broker/BrokerMain.swift - Privileged broker → Recovery fence: persists recovery
experiments/LockedUse/Sources/Broker/BrokerMain.swift - Privileged broker → Display guardian: coordinates protection
- Locked Use installer → Authorization rules: configures policy
experiments/LockedUse/Sources/Installer/InstallerMain.swift - Locked Use installer → Client approvals: configures approvals
experiments/LockedUse/Sources/Installer/InstallerMain.swift - Locked Use installer → Privileged broker: installs service