# apervmm/llm-use-automation architecture

> Architecture diagram of the GitHub repository apervmm/llm-use-automation, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-10-06). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/apervmm/llm-use-automation
- Repository: https://github.com/apervmm/llm-use-automation

## Overview

This repository automates browser workflows by first using an LLM-guided discovery session to record a capability, then replaying that saved capability deterministically with inputs. The main path runs through a CLI, browser surface, policy checks, and artifact storage; replay produces success, business-outcome, or failure results and can pause for operator handoff. README capabilities also include screenshots and run evidence, redaction, and checkpoint/outcome handling. Several links reflect explicit imports or README documentation; unsampled subsystem internals are not inferred.

Main entry points: [`src/commands/discover.py`](https://github.com/apervmm/llm-use-automation/blob/main/src/commands/discover.py) (Discovery command), [`src/surface/browser.py`](https://github.com/apervmm/llm-use-automation/blob/main/src/surface/browser.py) (Browser session), [`src/replay/executor.py`](https://github.com/apervmm/llm-use-automation/blob/main/src/replay/executor.py) (Replay executor), [`src/commands/replay.py`](https://github.com/apervmm/llm-use-automation/blob/main/src/commands/replay.py) (Replay command)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_entry["CLI workflows"]
  node_cli["CLI dispatch<br/>[cli.py]"]
  node_discover_cmd["Discovery command<br/>[discover.py]"]
  node_replay_cmd["Replay command<br/>[replay.py]"]
  node_operator_cli["Operator interaction<br/>[operator_cli.py]"]
end

subgraph group_discover["Discovery"]
  node_agent_loop["Agent loop<br/>[loop.py]"]
  node_llm["LLM client<br/>[llm_client.py]"]
  node_prompts["Prompts and tools<br/>[prompts.py]"]
  node_agent_utils["Agent actions<br/>[utils.py]"]
end

subgraph group_replay["Deterministic replay"]
  node_replay_engine["Replay executor<br/>[executor.py]"]
  node_replay_run["Replay run state<br/>[run.py]"]
  node_checkpoint["Checkpoint checks<br/>[checkpoint.py]"]
  node_outcomes["Replay outcomes<br/>[outcomes.py]"]
  node_escalation["Human handoff<br/>[handoff.py]"]
end

subgraph group_runtime["Browser and policy"]
  node_browser["Browser session<br/>[browser.py]"]
  node_perception["Page perception<br/>[perception.py]"]
  node_surface_types["Surface types<br/>[types.py]"]
  node_allowlist["Action allowlist<br/>[allowlist.py]"]
  node_redaction["Sensitive-data redaction<br/>[redaction.py]"]
end

subgraph group_records["Artifacts and evidence"]
  node_recorder["Capability recorder<br/>[recorder.py]"]
  node_schema["Capability schema<br/>[schema.py]"]
  node_store[("Artifact store<br/>[store.py]")]
  node_discover_evidence["Discovery evidence<br/>[logger.py]"]
end

node_user(("Operator"))
node_target_app(("Target web app"))

node_user -->|"invokes"| node_cli
node_cli -->|"dispatches"| node_discover_cmd
node_cli -->|"dispatches"| node_replay_cmd
node_discover_cmd -->|"opens session"| node_browser
node_discover_cmd -->|"starts discovery"| node_agent_loop
node_agent_loop -->|"requests decisions"| node_llm
node_agent_loop -->|"supplies tools"| node_prompts
node_agent_loop -->|"executes tool"| node_agent_utils
node_agent_utils -->|"performs actions"| node_browser
node_agent_utils -->|"takes snapshots"| node_perception
node_agent_loop -->|"requests handoff"| node_escalation
node_discover_cmd -->|"records capability"| node_recorder
node_discover_cmd -->|"checks completion"| node_checkpoint
node_recorder -->|"builds capability"| node_schema
node_discover_cmd -->|"saves capability"| node_store
node_discover_cmd -->|"logs run"| node_discover_evidence
node_replay_cmd -->|"loads capability"| node_store
node_replay_cmd -->|"opens session"| node_browser
node_replay_cmd -->|"starts replay"| node_replay_engine
node_replay_engine -->|"creates run"| node_replay_run
node_replay_engine -->|"executes steps"| node_browser
node_replay_engine -->|"checks checkpoint"| node_checkpoint
node_replay_engine -->|"returns result"| node_outcomes
node_replay_engine -->|"checks risky runs"| node_allowlist
node_replay_engine -->|"requests handoff"| node_escalation
node_replay_cmd -->|"configures operator"| node_operator_cli
node_discover_cmd -->|"configures operator"| node_operator_cli
node_escalation -->|"captures screenshot"| node_browser
node_operator_cli -.->|"requests decision"| node_user
node_browser -->|"checks actions"| node_allowlist
node_browser -.->|"drives browser"| node_target_app
node_browser -->|"returns action results"| node_surface_types
node_agent_loop -->|"uses page state"| node_surface_types
node_schema -->|"uses element references"| node_surface_types
node_replay_cmd -->|"redacts run data"| node_redaction

click node_cli "https://github.com/apervmm/llm-use-automation/blob/main/src/cli.py"
click node_discover_cmd "https://github.com/apervmm/llm-use-automation/blob/main/src/commands/discover.py"
click node_replay_cmd "https://github.com/apervmm/llm-use-automation/blob/main/src/commands/replay.py"
click node_agent_loop "https://github.com/apervmm/llm-use-automation/blob/main/src/agent/loop.py"
click node_llm "https://github.com/apervmm/llm-use-automation/blob/main/src/agent/llm_client.py"
click node_prompts "https://github.com/apervmm/llm-use-automation/blob/main/src/agent/prompts.py"
click node_agent_utils "https://github.com/apervmm/llm-use-automation/blob/main/src/agent/utils.py"
click node_browser "https://github.com/apervmm/llm-use-automation/blob/main/src/surface/browser.py"
click node_perception "https://github.com/apervmm/llm-use-automation/blob/main/src/surface/perception.py"
click node_surface_types "https://github.com/apervmm/llm-use-automation/blob/main/src/surface/types.py"
click node_allowlist "https://github.com/apervmm/llm-use-automation/blob/main/src/safety/allowlist.py"
click node_redaction "https://github.com/apervmm/llm-use-automation/blob/main/src/safety/redaction.py"
click node_recorder "https://github.com/apervmm/llm-use-automation/blob/main/src/artifact/recorder.py"
click node_schema "https://github.com/apervmm/llm-use-automation/blob/main/src/artifact/schema.py"
click node_store "https://github.com/apervmm/llm-use-automation/blob/main/src/artifact/store.py"
click node_discover_evidence "https://github.com/apervmm/llm-use-automation/blob/main/src/observability/logger.py"
click node_replay_engine "https://github.com/apervmm/llm-use-automation/blob/main/src/replay/executor.py"
click node_replay_run "https://github.com/apervmm/llm-use-automation/blob/main/src/replay/run.py"
click node_checkpoint "https://github.com/apervmm/llm-use-automation/blob/main/src/replay/checkpoint.py"
click node_outcomes "https://github.com/apervmm/llm-use-automation/blob/main/src/replay/outcomes.py"
click node_escalation "https://github.com/apervmm/llm-use-automation/blob/main/src/escalation/handoff.py"
click node_operator_cli "https://github.com/apervmm/llm-use-automation/blob/main/src/escalation/operator_cli.py"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_cli,node_discover_cmd,node_replay_cmd,node_operator_cli toneBlue
class node_agent_loop,node_llm,node_prompts,node_agent_utils toneAmber
class node_replay_engine,node_replay_run,node_checkpoint,node_outcomes,node_escalation toneMint
class node_browser,node_perception,node_surface_types,node_allowlist,node_redaction toneRose
class node_recorder,node_schema,node_store,node_discover_evidence,node_user,node_target_app toneIndigo
```

## Components

### CLI workflows

- [CLI dispatch](https://github.com/apervmm/llm-use-automation/blob/main/src/cli.py): `src/cli.py`
- [Discovery command](https://github.com/apervmm/llm-use-automation/blob/main/src/commands/discover.py): `src/commands/discover.py`
- [Replay command](https://github.com/apervmm/llm-use-automation/blob/main/src/commands/replay.py): `src/commands/replay.py`
- [Operator interaction](https://github.com/apervmm/llm-use-automation/blob/main/src/escalation/operator_cli.py): `src/escalation/operator_cli.py`

### Discovery

- [Agent loop](https://github.com/apervmm/llm-use-automation/blob/main/src/agent/loop.py): `src/agent/loop.py`
- [LLM client](https://github.com/apervmm/llm-use-automation/blob/main/src/agent/llm_client.py): `src/agent/llm_client.py`
- [Prompts and tools](https://github.com/apervmm/llm-use-automation/blob/main/src/agent/prompts.py): `src/agent/prompts.py`
- [Agent actions](https://github.com/apervmm/llm-use-automation/blob/main/src/agent/utils.py): `src/agent/utils.py`

### Deterministic replay

- [Replay executor](https://github.com/apervmm/llm-use-automation/blob/main/src/replay/executor.py): `src/replay/executor.py`
- [Replay run state](https://github.com/apervmm/llm-use-automation/blob/main/src/replay/run.py): `src/replay/run.py`
- [Checkpoint checks](https://github.com/apervmm/llm-use-automation/blob/main/src/replay/checkpoint.py): `src/replay/checkpoint.py`
- [Replay outcomes](https://github.com/apervmm/llm-use-automation/blob/main/src/replay/outcomes.py): `src/replay/outcomes.py`
- [Human handoff](https://github.com/apervmm/llm-use-automation/blob/main/src/escalation/handoff.py): `src/escalation/handoff.py`

### Browser and policy

- [Browser session](https://github.com/apervmm/llm-use-automation/blob/main/src/surface/browser.py): `src/surface/browser.py`
- [Page perception](https://github.com/apervmm/llm-use-automation/blob/main/src/surface/perception.py): `src/surface/perception.py`
- [Surface types](https://github.com/apervmm/llm-use-automation/blob/main/src/surface/types.py): `src/surface/types.py`
- [Action allowlist](https://github.com/apervmm/llm-use-automation/blob/main/src/safety/allowlist.py): `src/safety/allowlist.py`
- [Sensitive-data redaction](https://github.com/apervmm/llm-use-automation/blob/main/src/safety/redaction.py): `src/safety/redaction.py`

### Artifacts and evidence

- [Capability recorder](https://github.com/apervmm/llm-use-automation/blob/main/src/artifact/recorder.py): `src/artifact/recorder.py`
- [Capability schema](https://github.com/apervmm/llm-use-automation/blob/main/src/artifact/schema.py): `src/artifact/schema.py`
- [Artifact store](https://github.com/apervmm/llm-use-automation/blob/main/src/artifact/store.py): `src/artifact/store.py`
- [Discovery evidence](https://github.com/apervmm/llm-use-automation/blob/main/src/observability/logger.py): `src/observability/logger.py`

### Other components

- **Operator**
- **Target web app**

## Connections

- Operator → CLI dispatch: invokes
- CLI dispatch → Discovery command: dispatches (evidence: `src/cli.py`)
- CLI dispatch → Replay command: dispatches (evidence: `src/cli.py`)
- Discovery command → Browser session: opens session (evidence: `src/commands/discover.py`)
- Discovery command → Agent loop: starts discovery (evidence: `src/commands/discover.py`)
- Agent loop → LLM client: requests decisions (evidence: `src/agent/loop.py`)
- Agent loop → Prompts and tools: supplies tools (evidence: `src/agent/loop.py`)
- Agent loop → Agent actions: executes tool (evidence: `src/agent/loop.py`)
- Agent actions → Browser session: performs actions (evidence: `src/agent/utils.py`)
- Agent actions → Page perception: takes snapshots (evidence: `src/agent/utils.py`)
- Agent loop → Human handoff: requests handoff (evidence: `src/agent/loop.py`)
- Discovery command → Capability recorder: records capability (evidence: `src/commands/discover.py`)
- Discovery command → Checkpoint checks: checks completion (evidence: `src/commands/discover.py`)
- Capability recorder → Capability schema: builds capability (evidence: `src/artifact/recorder.py`)
- Discovery command → Artifact store: saves capability (evidence: `src/commands/discover.py`)
- Discovery command → Discovery evidence: logs run (evidence: `src/commands/discover.py`)
- Replay command → Artifact store: loads capability (evidence: `src/commands/replay.py`)
- Replay command → Browser session: opens session (evidence: `src/commands/replay.py`)
- Replay command → Replay executor: starts replay (evidence: `src/commands/replay.py`)
- Replay executor → Replay run state: creates run (evidence: `src/replay/executor.py`)
- Replay executor → Browser session: executes steps (evidence: `src/replay/executor.py`)
- Replay executor → Checkpoint checks: checks checkpoint (evidence: `src/replay/executor.py`)
- Replay executor → Replay outcomes: returns result (evidence: `src/replay/executor.py`)
- Replay executor → Action allowlist: checks risky runs (evidence: `src/replay/executor.py`)
- Replay executor → Human handoff: requests handoff (evidence: `src/replay/executor.py`)
- Replay command → Operator interaction: configures operator (evidence: `src/commands/replay.py`)
- Discovery command → Operator interaction: configures operator (evidence: `src/commands/discover.py`)
- Human handoff → Browser session: captures screenshot (evidence: `src/escalation/handoff.py`)
- Operator interaction → Operator: requests decision
- Browser session → Action allowlist: checks actions (evidence: `src/surface/browser.py`)
- Browser session → Target web app: drives browser
- Browser session → Surface types: returns action results (evidence: `src/surface/browser.py`)
- Agent loop → Surface types: uses page state (evidence: `src/agent/loop.py`)
- Capability schema → Surface types: uses element references (evidence: `src/artifact/schema.py`)
- Replay command → Sensitive-data redaction: redacts run data
