# injaneity/pi-computer-use architecture

> Architecture diagram of the GitHub repository injaneity/pi-computer-use, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-09-28). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/injaneity/pi-computer-use
- Repository: https://github.com/injaneity/pi-computer-use

## Overview

Pi Computer Use is a Pi extension that lets an agent discover desktop roots, observe and query interface state, then perform verified actions and wait for changes. The graph follows the documented tool workflow through the bridge to platform backends and native helpers, and includes browser-page support and state/outline handling. Platform helper internals are only partially sampled; edges to native implementations reflect the documented helper boundary and available excerpts, not an exhaustive account of each command.

Main entry points: [`src/bridge.ts`](https://github.com/injaneity/pi-computer-use/blob/main/src/bridge.ts) (Workflow bridge), [`src/platform/index.ts`](https://github.com/injaneity/pi-computer-use/blob/main/src/platform/index.ts) (Platform selection), [`extensions/computer-use.ts`](https://github.com/injaneity/pi-computer-use/blob/main/extensions/computer-use.ts) (Tool registration), [`native/windows/bridge-rs/src/main.rs`](https://github.com/injaneity/pi-computer-use/blob/main/native/windows/bridge-rs/src/main.rs) (Windows command bridge)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_agent["Agent interface"]
  node_extension["Tool registration<br/>[computer-use.ts]"]
end

subgraph group_core["Workflow core"]
  node_bridge["Workflow bridge<br/>[bridge.ts]"]
  node_config["Configuration<br/>[config.ts]"]
  node_state["Saved states<br/>[state.ts]"]
  node_scheduler["Resource scheduler<br/>[runtime.ts]"]
  node_outline["Outline processing<br/>[outline.ts]"]
  node_view["Rendered changes<br/>[view.ts]"]
end

subgraph group_platforms["Platform adapters"]
  node_platform["Platform selection<br/>[index.ts]"]
  node_macos["macOS backend<br/>[backend.ts]"]
  node_windows["Windows backend<br/>[backend.ts]"]
  node_linux["Linux backend<br/>[backend.ts]"]
  node_macosclient["macOS helper client<br/>[helper.ts]"]
  node_windowsclient["Windows helper client<br/>[helper.ts]"]
  node_linuxclient["Linux helper client<br/>[helper.ts]"]
end

subgraph group_native["Native helpers"]
  node_macoshelper["macOS native bridge<br/>[bridge.swift]"]
  node_winmain["Windows command bridge<br/>[main.rs]"]
  node_winuia["Windows UI access<br/>[uia.rs]"]
  node_wininput["Windows input<br/>[input.rs]"]
  node_linuxmain["Linux command bridge<br/>[main.rs]"]
  node_atspi["AT-SPI access<br/>[atspi.rs]"]
  node_x11["X11 support<br/>[x11.rs]"]
end

subgraph group_browser["Browser integration"]
  node_browserpages["Browser pages<br/>[cdp.ts]"]
end

node_pi(("Pi agent"))

node_pi -->|"invokes tools"| node_extension
node_extension -->|"loads settings"| node_config
node_extension -->|"dispatches tools"| node_bridge
node_bridge -->|"selects backend"| node_platform
node_bridge -->|"stores states"| node_state
node_bridge -->|"orders work"| node_scheduler
node_bridge -->|"processes outlines"| node_outline
node_bridge -->|"renders changes"| node_view
node_platform -->|"routes macOS"| node_macos
node_platform -->|"routes Windows"| node_windows
node_platform -->|"routes Linux"| node_linux
node_macos -->|"sends commands"| node_macosclient
node_windows -->|"sends commands"| node_windowsclient
node_linux -->|"sends commands"| node_linuxclient
node_macosclient -->|"uses helper"| node_macoshelper
node_windowsclient -->|"uses helper"| node_winmain
node_linuxclient -->|"uses helper"| node_linuxmain
node_winmain -->|"queries UI"| node_winuia
node_winmain -->|"performs input"| node_wininput
node_linuxmain -->|"queries UI"| node_atspi
node_linuxmain -->|"enriches roots"| node_x11
node_bridge -.->|"queries pages"| node_browserpages

click node_extension "https://github.com/injaneity/pi-computer-use/blob/main/extensions/computer-use.ts"
click node_bridge "https://github.com/injaneity/pi-computer-use/blob/main/src/bridge.ts"
click node_config "https://github.com/injaneity/pi-computer-use/blob/main/src/config.ts"
click node_state "https://github.com/injaneity/pi-computer-use/blob/main/src/state.ts"
click node_scheduler "https://github.com/injaneity/pi-computer-use/blob/main/src/runtime.ts"
click node_outline "https://github.com/injaneity/pi-computer-use/blob/main/src/outline.ts"
click node_view "https://github.com/injaneity/pi-computer-use/blob/main/src/view.ts"
click node_platform "https://github.com/injaneity/pi-computer-use/blob/main/src/platform/index.ts"
click node_macos "https://github.com/injaneity/pi-computer-use/blob/main/src/platform/macos/backend.ts"
click node_windows "https://github.com/injaneity/pi-computer-use/blob/main/src/platform/windows/backend.ts"
click node_linux "https://github.com/injaneity/pi-computer-use/blob/main/src/platform/linux/backend.ts"
click node_macosclient "https://github.com/injaneity/pi-computer-use/blob/main/src/platform/macos/helper.ts"
click node_windowsclient "https://github.com/injaneity/pi-computer-use/blob/main/src/platform/windows/helper.ts"
click node_linuxclient "https://github.com/injaneity/pi-computer-use/blob/main/src/platform/linux/helper.ts"
click node_macoshelper "https://github.com/injaneity/pi-computer-use/blob/main/native/macos/bridge.swift"
click node_winmain "https://github.com/injaneity/pi-computer-use/blob/main/native/windows/bridge-rs/src/main.rs"
click node_winuia "https://github.com/injaneity/pi-computer-use/blob/main/native/windows/bridge-rs/src/uia.rs"
click node_wininput "https://github.com/injaneity/pi-computer-use/blob/main/native/windows/bridge-rs/src/input.rs"
click node_linuxmain "https://github.com/injaneity/pi-computer-use/blob/main/native/linux/bridge-rs/src/main.rs"
click node_atspi "https://github.com/injaneity/pi-computer-use/blob/main/native/linux/bridge-rs/src/atspi.rs"
click node_x11 "https://github.com/injaneity/pi-computer-use/blob/main/native/linux/bridge-rs/src/x11.rs"
click node_browserpages "https://github.com/injaneity/pi-computer-use/blob/main/src/cdp.ts"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_extension toneBlue
class node_bridge,node_config,node_state,node_scheduler,node_outline,node_view toneAmber
class node_platform,node_macos,node_windows,node_linux,node_macosclient,node_windowsclient,node_linuxclient toneMint
class node_macoshelper,node_winmain,node_winuia,node_wininput,node_linuxmain,node_atspi,node_x11 toneRose
class node_browserpages,node_pi toneIndigo
```

## Components

### Agent interface

- [Tool registration](https://github.com/injaneity/pi-computer-use/blob/main/extensions/computer-use.ts): `extensions/computer-use.ts`

### Workflow core

- [Workflow bridge](https://github.com/injaneity/pi-computer-use/blob/main/src/bridge.ts): `src/bridge.ts`
- [Configuration](https://github.com/injaneity/pi-computer-use/blob/main/src/config.ts): `src/config.ts`
- [Saved states](https://github.com/injaneity/pi-computer-use/blob/main/src/state.ts): `src/state.ts`
- [Resource scheduler](https://github.com/injaneity/pi-computer-use/blob/main/src/runtime.ts): `src/runtime.ts`
- [Outline processing](https://github.com/injaneity/pi-computer-use/blob/main/src/outline.ts): `src/outline.ts`
- [Rendered changes](https://github.com/injaneity/pi-computer-use/blob/main/src/view.ts): `src/view.ts`

### Platform adapters

- [Platform selection](https://github.com/injaneity/pi-computer-use/blob/main/src/platform/index.ts): `src/platform/index.ts`
- [macOS backend](https://github.com/injaneity/pi-computer-use/blob/main/src/platform/macos/backend.ts): `src/platform/macos/backend.ts`
- [Windows backend](https://github.com/injaneity/pi-computer-use/blob/main/src/platform/windows/backend.ts): `src/platform/windows/backend.ts`
- [Linux backend](https://github.com/injaneity/pi-computer-use/blob/main/src/platform/linux/backend.ts): `src/platform/linux/backend.ts`
- [macOS helper client](https://github.com/injaneity/pi-computer-use/blob/main/src/platform/macos/helper.ts): `src/platform/macos/helper.ts`
- [Windows helper client](https://github.com/injaneity/pi-computer-use/blob/main/src/platform/windows/helper.ts): `src/platform/windows/helper.ts`
- [Linux helper client](https://github.com/injaneity/pi-computer-use/blob/main/src/platform/linux/helper.ts): `src/platform/linux/helper.ts`

### Native helpers

- [macOS native bridge](https://github.com/injaneity/pi-computer-use/blob/main/native/macos/bridge.swift): `native/macos/bridge.swift`
- [Windows command bridge](https://github.com/injaneity/pi-computer-use/blob/main/native/windows/bridge-rs/src/main.rs): `native/windows/bridge-rs/src/main.rs`
- [Windows UI access](https://github.com/injaneity/pi-computer-use/blob/main/native/windows/bridge-rs/src/uia.rs): `native/windows/bridge-rs/src/uia.rs`
- [Windows input](https://github.com/injaneity/pi-computer-use/blob/main/native/windows/bridge-rs/src/input.rs): `native/windows/bridge-rs/src/input.rs`
- [Linux command bridge](https://github.com/injaneity/pi-computer-use/blob/main/native/linux/bridge-rs/src/main.rs): `native/linux/bridge-rs/src/main.rs`
- [AT-SPI access](https://github.com/injaneity/pi-computer-use/blob/main/native/linux/bridge-rs/src/atspi.rs): `native/linux/bridge-rs/src/atspi.rs`
- [X11 support](https://github.com/injaneity/pi-computer-use/blob/main/native/linux/bridge-rs/src/x11.rs): `native/linux/bridge-rs/src/x11.rs`

### Browser integration

- [Browser pages](https://github.com/injaneity/pi-computer-use/blob/main/src/cdp.ts): `src/cdp.ts`

### Other components

- **Pi agent**

## Connections

- Pi agent → Tool registration: invokes tools
- Tool registration → Configuration: loads settings
- Tool registration → Workflow bridge: dispatches tools
- Workflow bridge → Platform selection: selects backend
- Workflow bridge → Saved states: stores states
- Workflow bridge → Resource scheduler: orders work
- Workflow bridge → Outline processing: processes outlines
- Workflow bridge → Rendered changes: renders changes
- Platform selection → macOS backend: routes macOS
- Platform selection → Windows backend: routes Windows
- Platform selection → Linux backend: routes Linux
- macOS backend → macOS helper client: sends commands
- Windows backend → Windows helper client: sends commands
- Linux backend → Linux helper client: sends commands
- macOS helper client → macOS native bridge: uses helper
- Windows helper client → Windows command bridge: uses helper
- Linux helper client → Linux command bridge: uses helper
- Windows command bridge → Windows UI access: queries UI
- Windows command bridge → Windows input: performs input
- Linux command bridge → AT-SPI access: queries UI
- Linux command bridge → X11 support: enriches roots
- Workflow bridge → Browser pages: queries pages