# sm5190/interface-computer-use architecture

> Architecture diagram of the GitHub repository sm5190/interface-computer-use, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-09-26). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/sm5190/interface-computer-use
- Repository: https://github.com/sm5190/interface-computer-use
- Video tour (about a minute): https://gitdiagram.com/sm5190/interface-computer-use/video

## Overview

Interface Computer Use automates browser workflows by discovering a task with an LLM, compiling the observed trajectory into a typed capability, then replaying that capability deterministically under policy controls. Operators initiate discovery, compilation, or replay through the CLI. Replay and discovery share browser interaction, target grounding, checkpoint evaluation, safety gating, and evidence capture. The repository also includes LegacyBank, a synthetic banking UI target with member, account, and transfer operations. README capabilities include human approval and structured outcomes; the excerpts show policy approval gates and outcome/evidence handling, but do not expose the complete handoff wiring.

Main entry points: [`src/cua/discovery/engine.py`](https://github.com/sm5190/interface-computer-use/blob/main/src/cua/discovery/engine.py) (Discovery Engine), [`src/cua/replay/engine.py`](https://github.com/sm5190/interface-computer-use/blob/main/src/cua/replay/engine.py) (Replay Engine), [`src/cua/cli.py`](https://github.com/sm5190/interface-computer-use/blob/main/src/cua/cli.py) (CLI), [`src/cua/safety/gate.py`](https://github.com/sm5190/interface-computer-use/blob/main/src/cua/safety/gate.py) (Action Gate)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_entry["Operator Entry"]
  node_cli["CLI<br/>[cli.py]"]
end

subgraph group_discovery["Workflow Discovery"]
  node_discovery_engine["Discovery Engine<br/>[engine.py]"]
  node_discovery_model["Discovery Model<br/>[gemini.py]"]
  node_evidence["Evidence Recorder<br/>[recorder.py]"]
end

subgraph group_capability["Capability Artifacts"]
  node_compiler["Capability Compiler<br/>[compiler.py]"]
  node_domain_models["Typed Domain Models"]
end

subgraph group_runtime["Execution Runtime"]
  node_replay_engine["Replay Engine<br/>[engine.py]"]
  node_bootstrap["Session Bootstrap<br/>[bootstrap.py]"]
  node_action_gate["Action Gate<br/>[gate.py]"]
  node_application_policy["Application Policy<br/>[policy.py]"]
  node_risk["Risk Classification<br/>[risk.py]"]
  node_redaction["Sensitive Data Redaction<br/>[redaction.py]"]
  node_resolver["Target Resolver<br/>[resolver.py]"]
  node_checkpoints["Checkpoint Evaluator<br/>[checkpoints.py]"]
  node_surface["Playwright Surface"]
  node_human_approval(("Human Approval"))
end

subgraph group_target["LegacyBank Target"]
  node_legacy_app["Banking Console<br/>[app.py]"]
  node_bank_db[("Banking Database<br/>[db.py]")]
end

node_operator(("Operator"))
node_browser(("Browser"))

node_operator -->|"runs commands"| node_cli
node_cli -->|"starts discovery"| node_discovery_engine
node_cli -->|"compiles trajectory"| node_compiler
node_cli -->|"starts replay"| node_replay_engine
node_discovery_engine -->|"requests decisions"| node_discovery_model
node_discovery_engine -->|"observes and acts"| node_surface
node_discovery_engine -->|"grounds targets"| node_resolver
node_discovery_engine -->|"checks actions"| node_action_gate
node_discovery_engine -->|"evaluates effects"| node_checkpoints
node_discovery_engine -->|"records run"| node_evidence
node_compiler -->|"builds artifact"| node_domain_models
node_cli -->|"validates inputs"| node_domain_models
node_replay_engine -->|"observes and acts"| node_surface
node_replay_engine -->|"grounds targets"| node_resolver
node_replay_engine -->|"checks actions"| node_action_gate
node_replay_engine -->|"evaluates effects"| node_checkpoints
node_replay_engine -.->|"initializes session"| node_bootstrap
node_action_gate -->|"consults rules"| node_application_policy
node_action_gate -->|"classifies risk"| node_risk
node_action_gate -->|"requires approval"| node_human_approval
node_checkpoints -->|"resolves controls"| node_resolver
node_checkpoints -->|"reads page state"| node_surface
node_surface -->|"controls browser"| node_browser
node_browser -->|"loads and operates"| node_legacy_app
node_legacy_app -->|"reads and writes"| node_bank_db
node_evidence -->|"sanitizes records"| node_redaction

click node_cli "https://github.com/sm5190/interface-computer-use/blob/main/src/cua/cli.py"
click node_discovery_engine "https://github.com/sm5190/interface-computer-use/blob/main/src/cua/discovery/engine.py"
click node_discovery_model "https://github.com/sm5190/interface-computer-use/blob/main/src/cua/discovery/gemini.py"
click node_evidence "https://github.com/sm5190/interface-computer-use/blob/main/src/cua/evidence/recorder.py"
click node_compiler "https://github.com/sm5190/interface-computer-use/blob/main/src/cua/compiler/compiler.py"
click node_domain_models "https://github.com/sm5190/interface-computer-use/tree/main/src/cua/domain"
click node_replay_engine "https://github.com/sm5190/interface-computer-use/blob/main/src/cua/replay/engine.py"
click node_bootstrap "https://github.com/sm5190/interface-computer-use/blob/main/src/cua/replay/bootstrap.py"
click node_action_gate "https://github.com/sm5190/interface-computer-use/blob/main/src/cua/safety/gate.py"
click node_application_policy "https://github.com/sm5190/interface-computer-use/blob/main/src/cua/safety/policy.py"
click node_risk "https://github.com/sm5190/interface-computer-use/blob/main/src/cua/safety/risk.py"
click node_redaction "https://github.com/sm5190/interface-computer-use/blob/main/src/cua/safety/redaction.py"
click node_resolver "https://github.com/sm5190/interface-computer-use/blob/main/src/cua/grounding/resolver.py"
click node_checkpoints "https://github.com/sm5190/interface-computer-use/blob/main/src/cua/grounding/checkpoints.py"
click node_surface "https://github.com/sm5190/interface-computer-use/blob/main/src/cua/surfaces/playwright_surface.py"
click node_legacy_app "https://github.com/sm5190/interface-computer-use/blob/main/demo_app/app.py"
click node_bank_db "https://github.com/sm5190/interface-computer-use/blob/main/demo_app/db.py"
click node_human_approval "https://github.com/sm5190/interface-computer-use/tree/main/src/cua/handoff"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_cli,node_browser toneBlue
class node_discovery_engine,node_discovery_model,node_evidence toneAmber
class node_compiler,node_domain_models toneMint
class node_replay_engine,node_bootstrap,node_action_gate,node_application_policy,node_risk,node_redaction,node_resolver,node_checkpoints,node_surface,node_human_approval toneRose
class node_legacy_app,node_bank_db,node_operator toneIndigo
```

## Components

### Operator Entry

- [CLI](https://github.com/sm5190/interface-computer-use/blob/main/src/cua/cli.py): `src/cua/cli.py`

### Workflow Discovery

- [Discovery Engine](https://github.com/sm5190/interface-computer-use/blob/main/src/cua/discovery/engine.py): `src/cua/discovery/engine.py`
- [Discovery Model](https://github.com/sm5190/interface-computer-use/blob/main/src/cua/discovery/gemini.py): `src/cua/discovery/gemini.py`
- [Evidence Recorder](https://github.com/sm5190/interface-computer-use/blob/main/src/cua/evidence/recorder.py): `src/cua/evidence/recorder.py`

### Capability Artifacts

- [Capability Compiler](https://github.com/sm5190/interface-computer-use/blob/main/src/cua/compiler/compiler.py): `src/cua/compiler/compiler.py`
- [Typed Domain Models](https://github.com/sm5190/interface-computer-use/tree/main/src/cua/domain): `src/cua/domain`

### Execution Runtime

- [Replay Engine](https://github.com/sm5190/interface-computer-use/blob/main/src/cua/replay/engine.py): `src/cua/replay/engine.py`
- [Session Bootstrap](https://github.com/sm5190/interface-computer-use/blob/main/src/cua/replay/bootstrap.py): `src/cua/replay/bootstrap.py`
- [Action Gate](https://github.com/sm5190/interface-computer-use/blob/main/src/cua/safety/gate.py): `src/cua/safety/gate.py`
- [Application Policy](https://github.com/sm5190/interface-computer-use/blob/main/src/cua/safety/policy.py): `src/cua/safety/policy.py`
- [Risk Classification](https://github.com/sm5190/interface-computer-use/blob/main/src/cua/safety/risk.py): `src/cua/safety/risk.py`
- [Sensitive Data Redaction](https://github.com/sm5190/interface-computer-use/blob/main/src/cua/safety/redaction.py): `src/cua/safety/redaction.py`
- [Target Resolver](https://github.com/sm5190/interface-computer-use/blob/main/src/cua/grounding/resolver.py): `src/cua/grounding/resolver.py`
- [Checkpoint Evaluator](https://github.com/sm5190/interface-computer-use/blob/main/src/cua/grounding/checkpoints.py): `src/cua/grounding/checkpoints.py`
- [Playwright Surface](https://github.com/sm5190/interface-computer-use/blob/main/src/cua/surfaces/playwright_surface.py): `src/cua/surfaces/playwright_surface.py`
- [Human Approval](https://github.com/sm5190/interface-computer-use/tree/main/src/cua/handoff): `src/cua/handoff`

### LegacyBank Target

- [Banking Console](https://github.com/sm5190/interface-computer-use/blob/main/demo_app/app.py): `demo_app/app.py`
- [Banking Database](https://github.com/sm5190/interface-computer-use/blob/main/demo_app/db.py): `demo_app/db.py`

### Other components

- **Operator**
- **Browser**

## Connections

- Operator → CLI: runs commands
- CLI → Discovery Engine: starts discovery
- CLI → Capability Compiler: compiles trajectory
- CLI → Replay Engine: starts replay
- Discovery Engine → Discovery Model: requests decisions
- Discovery Engine → Playwright Surface: observes and acts
- Discovery Engine → Target Resolver: grounds targets
- Discovery Engine → Action Gate: checks actions
- Discovery Engine → Checkpoint Evaluator: evaluates effects
- Discovery Engine → Evidence Recorder: records run
- Capability Compiler → Typed Domain Models: builds artifact
- CLI → Typed Domain Models: validates inputs
- Replay Engine → Playwright Surface: observes and acts
- Replay Engine → Target Resolver: grounds targets
- Replay Engine → Action Gate: checks actions
- Replay Engine → Checkpoint Evaluator: evaluates effects
- Replay Engine → Session Bootstrap: initializes session
- Action Gate → Application Policy: consults rules
- Action Gate → Risk Classification: classifies risk
- Action Gate → Human Approval: requires approval
- Checkpoint Evaluator → Target Resolver: resolves controls
- Checkpoint Evaluator → Playwright Surface: reads page state
- Playwright Surface → Browser: controls browser
- Browser → Banking Console: loads and operates
- Banking Console → Banking Database: reads and writes
- Evidence Recorder → Sensitive Data Redaction: sanitizes records