# lmnr-ai/index architecture

> Architecture diagram of the GitHub repository lmnr-ai/index, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-04-17). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/lmnr-ai/index
- Repository: https://github.com/lmnr-ai/index

## Overview

##### Project type and purpose
- This is an open‑source Python library (pip package) called “Index” that implements an autonomous browser agent.
- Purpose: drive a real or headless browser (via Playwright/CDP), issue actions (click, type, scrape) based on LLM reasoning, and return structured results.
- It can be used locally (as a Python client), remotely via the Laminar‑hosted API, or connected to an existing CDP endpoint.

##### File structure
Top‑level:
- index/        ← main Python package
- static/       ← images and assets for README
- pyproject.toml, LICENSE, README.md
- .github/workflows/publish.yml ← CI/CD for publishing

index/ subpackages:
- agent/        ← high‑level orchestration: Agent class, prompts, message management, result models
- browser/      ← low‑level browser control: launch/connect CDP, DOM detection, utilities, JS snippets
- controller/   ← command executor: maps agent decisions to browser actions (default_actions.py)
- llm/          ← provider adapters: unified interface to OpenAI, Anthropic, Bedrock, etc.

##### Key insights from README
- Core flow: user code → Agent.run(prompt) → Agent uses LLM provider → Controller issues commands → Browser module executes via Playwright/CDP → observability/tracing to Laminar.
- Supports streaming outputs, remote CDP connections, custom viewport, and full tracing on Laminar platform.
- Exhibits a plugin architecture for LLM providers and a clear separation between decision‑making (agent), action execution (controller), and browser interfacing.

1. User Application / Client
2. Agent Module
3. LLM Providers
4. Controller Module
5. Browser Module (Playwright/CDP)
6. Observability/Tracing (Laminar Platform)
7. External Services:
- LLM Endpoints (OpenAI, Anthropic, Bedrock)
- Remote CDP Endpoint (optional)

- User → Agent.run(prompt)
- Agent ↔ LLM Provider Adapter → External LLM Endpoint
- Agent → Controller: sequence of actions/commands
- Controller → Browser Module: CDP commands (click, type, evaluate JS)
- Browser Module → Controller: DOM snapshots, action results
- Throughout, Observability Hook captures events and streams to Laminar
- (Optional) Browser Module ←→ Remote CDP Endpoint

c. Architectural patterns & principles:
- Separation of Concerns: agent (reasoning), controller (action mapping), browser (execution).
- Plugin Pattern: llm/providers directory allows adding new LLM adapters.
- Command Pattern: controller/default_actions defines discrete, composable browser commands.
- Asynchronous Event Loop: all modules use asyncio for non‑blocking I/O.

d. Technologies & frameworks:
- Python 3.x, asyncio
- Playwright (CDP)
- JavaScript snippet (findVisibleInteractiveElements.js)
- LLM APIs (Anthropic, OpenAI, AWS Bedrock)
- Laminar observability SDK (lmnr)
- PyProject/TOML packaging

- Both local and hosted API usage paths.
- Plugin architecture for LLM providers.
- Command flow: prompt → reasoning → actions → browser execution → result.
- Observability/tracing as a cross‑cutting concern.
- Use consistent naming matching the file tree (e.g., “index/llm/providers/openai.py”).

## Diagram (Mermaid)

```mermaid
flowchart TD
    %% User Layer
    subgraph "User"
        direction TB
        UserApp["User Application / Client"]:::external
        HostedAPI["Hosted API (AsyncLaminarClient)"]:::external
    end

    %% Agent Module
    subgraph "index/agent"
        direction TB
        Agent["Agent Module"]:::module
    end

    %% Core LLM Interface
    CoreLLM["Core LLM Interface\n(index/llm/llm.py)"]:::module

    %% LLM Provider Adapters
    subgraph "index/llm/providers"
        direction TB
        OpenAIAdapter["OpenAI Adapter"]:::plugin
        AnthropicAdapter["Anthropic Adapter"]:::plugin
        BedrockAdapter["Anthropic Bedrock Adapter"]:::plugin
    end

    %% Controller Module
    subgraph "index/controller"
        direction TB
        Controller["Controller Module"]:::module
    end

    %% Browser Module
    subgraph "index/browser"
        direction TB
        Browser["Browser Module"]:::module
    end

    %% External Services
    subgraph "External Services"
        direction TB
        LLMEndpoints["LLM Endpoints\n(OpenAI, Anthropic, Bedrock)"]:::external
        RemoteCDP["Remote CDP Endpoint"]:::external
    end

    %% Observability
    Observability["Observability & Tracing\n(Laminar Platform)"]:::external

    %% Data Flows
    UserApp -->|"prompt & context"| Agent
    HostedAPI -->|"prompt & context"| Agent
    Agent -->|"llm interface"| CoreLLM
    CoreLLM -->|"select adapter"| OpenAIAdapter
    CoreLLM -->|"select adapter"| AnthropicAdapter
    CoreLLM -->|"select adapter"| BedrockAdapter
    OpenAIAdapter -->|"API call"| LLMEndpoints
    AnthropicAdapter -->|"API call"| LLMEndpoints
    BedrockAdapter -->|"API call"| LLMEndpoints
    Agent -->|"action commands"| Controller
    Controller -->|"CDP commands"| Browser
    Browser -->|"DOM events & responses"| Controller
    Browser -.->|"optional remote"| RemoteCDP

    %% Observability Hooks
    Agent -->|"tracing events"| Observability
    Controller -->|"tracing events"| Observability
    Browser -->|"tracing events"| Observability
    OpenAIAdapter -->|"tracing events"| Observability
    AnthropicAdapter -->|"tracing events"| Observability
    BedrockAdapter -->|"tracing events"| Observability

    %% Click Events
    click Agent "https://github.com/lmnr-ai/index/tree/main/index/agent/"
    click CoreLLM "https://github.com/lmnr-ai/index/blob/main/index/llm/llm.py"
    click OpenAIAdapter "https://github.com/lmnr-ai/index/blob/main/index/llm/providers/openai.py"
    click AnthropicAdapter "https://github.com/lmnr-ai/index/blob/main/index/llm/providers/anthropic.py"
    click BedrockAdapter "https://github.com/lmnr-ai/index/blob/main/index/llm/providers/anthropic_bedrock.py"
    click Controller "https://github.com/lmnr-ai/index/tree/main/index/controller/"
    click Browser "https://github.com/lmnr-ai/index/tree/main/index/browser/"

    %% Styles
    classDef module fill:#1f77b4,stroke:#000,color:#fff;
    classDef plugin fill:#2ca02c,stroke:#000,color:#fff;
    classDef external fill:#7f7f7f,stroke:#000,color:#fff;
```
