How apervmm/llm-use-automation works
This repository automates browser workflows by first using an LLM-guided discovery session to record a capability, then replaying that saved capability deterministically with inputs. The main path runs through a CLI, browser surface, policy checks, and artifact storage; replay produces success, business-outcome, or failure results and can pause for operator handoff. README capabilities also include screenshots and run evidence, redaction, and checkpoint/outcome handling. Several links reflect explicit imports or README documentation; unsampled subsystem internals are not inferred.
Main entry points: discover.py Discovery command · browser.py Browser session · executor.py Replay executor · replay.py Replay command
Components 24
CLI workflows
- CLI dispatch
src/cli.py - Discovery command
src/commands/discover.py - Replay command
src/commands/replay.py - Operator interaction
src/escalation/operator_cli.py
Discovery
- Agent loop
src/agent/loop.py - LLM client
src/agent/llm_client.py - Prompts and tools
src/agent/prompts.py - Agent actions
src/agent/utils.py
Deterministic replay
- Replay executor
src/replay/executor.py - Replay run state
src/replay/run.py - Checkpoint checks
src/replay/checkpoint.py - Replay outcomes
src/replay/outcomes.py - Human handoff
src/escalation/handoff.py
Browser and policy
- Browser session
src/surface/browser.py - Page perception
src/surface/perception.py - Surface types
src/surface/types.py - Action allowlist
src/safety/allowlist.py - Sensitive-data redaction
src/safety/redaction.py
Artifacts and evidence
- Capability recorder
src/artifact/recorder.py - Capability schema
src/artifact/schema.py - Artifact store
src/artifact/store.py - Discovery evidence
src/observability/logger.py
Other components
- Operator
- Target web app
Connections 35
- Operator → CLI dispatch: invokes
- CLI dispatch → Discovery command: dispatches
src/cli.py - CLI dispatch → Replay command: dispatches
src/cli.py - Discovery command → Browser session: opens session
src/commands/discover.py - Discovery command → Agent loop: starts discovery
src/commands/discover.py - Agent loop → LLM client: requests decisions
src/agent/loop.py - Agent loop → Prompts and tools: supplies tools
src/agent/loop.py - Agent loop → Agent actions: executes tool
src/agent/loop.py - Agent actions → Browser session: performs actions
src/agent/utils.py - Agent actions → Page perception: takes snapshots
src/agent/utils.py - Agent loop → Human handoff: requests handoff
src/agent/loop.py - Discovery command → Capability recorder: records capability
src/commands/discover.py - Discovery command → Checkpoint checks: checks completion
src/commands/discover.py - Capability recorder → Capability schema: builds capability
src/artifact/recorder.py - Discovery command → Artifact store: saves capability
src/commands/discover.py - Discovery command → Discovery evidence: logs run
src/commands/discover.py - Replay command → Artifact store: loads capability
src/commands/replay.py - Replay command → Browser session: opens session
src/commands/replay.py - Replay command → Replay executor: starts replay
src/commands/replay.py - Replay executor → Replay run state: creates run
src/replay/executor.py - Replay executor → Browser session: executes steps
src/replay/executor.py - Replay executor → Checkpoint checks: checks checkpoint
src/replay/executor.py - Replay executor → Replay outcomes: returns result
src/replay/executor.py - Replay executor → Action allowlist: checks risky runs
src/replay/executor.py - Replay executor → Human handoff: requests handoff
src/replay/executor.py - Replay command → Operator interaction: configures operator
src/commands/replay.py - Discovery command → Operator interaction: configures operator
src/commands/discover.py - Human handoff → Browser session: captures screenshot
src/escalation/handoff.py - Operator interaction → Operator: requests decision
- Browser session → Action allowlist: checks actions
src/surface/browser.py - Browser session → Target web app: drives browser
- Browser session → Surface types: returns action results
src/surface/browser.py - Agent loop → Surface types: uses page state
src/agent/loop.py - Capability schema → Surface types: uses element references
src/artifact/schema.py - Replay command → Sensitive-data redaction: redacts run data