How lmnr-ai/index works
Project type and purpose
• This is an open‑source Python library (pip package) called “Index” that implements an autonomous browser agent.
• Purpose: drive a real or headless browser (via Playwright/CDP), issue actions (click, type, scrape) based on LLM reasoning, and return structured results.
• It can be used locally (as a Python client), remotely via the Laminar‑hosted API, or connected to an existing CDP endpoint.
File structure
Top‑level:
• index/ ← main Python package
• static/ ← images and assets for README
• pyproject.toml, LICENSE, README.md
• .github/workflows/publish.yml ← CI/CD for publishing
index/ subpackages:
• agent/ ← high‑level orchestration: Agent class, prompts, message management, result models
• browser/ ← low‑level browser control: launch/connect CDP, DOM detection, utilities, JS snippets
• controller/ ← command executor: maps agent decisions to browser actions (default_actions.py)
• llm/ ← provider adapters: unified interface to OpenAI, Anthropic, Bedrock, etc.
Key insights from README
• Core flow: user code → Agent.run(prompt) → Agent uses LLM provider → Controller issues commands → Browser module executes via Playwright/CDP → observability/tracing to Laminar.
• Supports streaming outputs, remote CDP connections, custom viewport, and full tracing on Laminar platform.
• Exhibits a plugin architecture for LLM providers and a clear separation between decision‑making (agent), action execution (controller), and browser interfacing.
1. User Application / Client
2. Agent Module
3. LLM Providers
4. Controller Module
5. Browser Module (Playwright/CDP)
6. Observability/Tracing (Laminar Platform)
7. External Services:
• LLM Endpoints (OpenAI, Anthropic, Bedrock)
• Remote CDP Endpoint (optional)
• User → Agent.run(prompt)
• Agent ↔ LLM Provider Adapter → External LLM Endpoint
• Agent → Controller: sequence of actions/commands
• Controller → Browser Module: CDP commands (click, type, evaluate JS)
• Browser Module → Controller: DOM snapshots, action results
• Throughout, Observability Hook captures events and streams to Laminar
• (Optional) Browser Module ←→ Remote CDP Endpoint
c. Architectural patterns & principles:
• Separation of Concerns: agent (reasoning), controller (action mapping), browser (execution).
• Plugin Pattern: llm/providers directory allows adding new LLM adapters.
• Command Pattern: controller/default_actions defines discrete, composable browser commands.
• Asynchronous Event Loop: all modules use asyncio for non‑blocking I/O.
d. Technologies & frameworks:
• Python 3.x, asyncio
• Playwright (CDP)
• JavaScript snippet (findVisibleInteractiveElements.js)
• LLM APIs (Anthropic, OpenAI, AWS Bedrock)
• Laminar observability SDK (lmnr)
• PyProject/TOML packaging
• Both local and hosted API usage paths.
• Plugin architecture for LLM providers.
• Command flow: prompt → reasoning → actions → browser execution → result.
• Observability/tracing as a cross‑cutting concern.
• Use consistent naming matching the file tree (e.g., “index/llm/providers/openai.py”).