# mtehabsim/screenpilot architecture

> Architecture diagram of the GitHub repository mtehabsim/screenpilot, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-05-21). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/mtehabsim/screenpilot
- Repository: https://github.com/mtehabsim/screenpilot

## Overview

##### Project type and purpose
- ScreenPilot is a Python-based “MCP server” (a micro-service) that exposes a suite of GUI automation tools (screen capture, mouse, keyboard, etc.) to a large-language-model client (e.g., Claude AI).
- It is not a web app or compiler: it’s a standalone automation service that integrates with an LLM via the MCP (Model-Control-Protocol) interface.

##### File structure
- Top-level entries:
- main.py         ← service entrypoint, starts the MCP server
- config.py       ← configuration loader for MCP registration
- requirements.txt
- core/           ← the heart of the toolkit
- utils.py        ← shared helpers
- core/ contains:
- base_tool.py        ← abstract base for all tools
- screen_capture.py   ← screenshot and image‐analysis APIs
- mouse.py            ← move/click methods
- keyboard.py         ← typing and hotkey methods
- scroll.py           ← scrolling control
- element.py          ← element detection / waiting
- action_sequence.py  ← orchestration of multiple tool calls

##### README for additional insights
- Describes how ScreenPilot plugs into the Claude AI desktop via a JSON config.
- Lists all available tools (screen, mouse, keyboard, scroll, element detection, sequences).

## Diagram (Mermaid)

```mermaid
flowchart TB
    %% Client Layer
    LLM["LLM Client"]:::external

    %% Service Layer
    subgraph "Service Layer"
        direction TB
        MCP["MCP Server"]:::service
        Config["Config Loader"]:::service
    end

    %% Toolkit Layer
    subgraph "Toolkit Layer"
        direction TB
        subgraph "Core Toolkit"
            direction TB
            Base["BaseTool"]:::core
            Screen["ScreenCapture"]:::core
            Mouse["MouseControl"]:::core
            Keyboard["KeyboardActions"]:::core
            Scroll["Scrolling"]:::core
            Element["ElementDetection"]:::core
            Sequence["ActionSequence"]:::core
        end
        Utils["Utils Module"]:::service
    end

    %% Infrastructure
    Infra["OS/Hardware"]:::infra

    %% Connections
    LLM -->|"JSON-over-stdio (commands)"| MCP
    MCP -->|"register tools"| Config
    MCP -->|"dispatch command"| Base
    Base -->|extends| Screen
    Base -->|extends| Mouse
    Base -->|extends| Keyboard
    Base -->|extends| Scroll
    Base -->|extends| Element
    Base -->|extends| Sequence
    MCP -->|"uses helper"| Utils
    Screen -->|"capture screenshot"| Infra
    Mouse -->|"execute click/move"| Infra
    Keyboard -->|"type/hotkey"| Infra
    Scroll -->|"scroll action"| Infra
    Element -->|"detect/wait"| Infra
    Sequence -->|"orchestrate tools"| Infra

    %% Click Events
    click MCP "https://github.com/mtehabsim/screenpilot/blob/main/main.py"
    click Config "https://github.com/mtehabsim/screenpilot/blob/main/config.py"
    click Utils "https://github.com/mtehabsim/screenpilot/blob/main/utils.py"
    click Base "https://github.com/mtehabsim/screenpilot/blob/main/core/base_tool.py"
    click Screen "https://github.com/mtehabsim/screenpilot/blob/main/core/screen_capture.py"
    click Mouse "https://github.com/mtehabsim/screenpilot/blob/main/core/mouse.py"
    click Keyboard "https://github.com/mtehabsim/screenpilot/blob/main/core/keyboard.py"
    click Scroll "https://github.com/mtehabsim/screenpilot/blob/main/core/scroll.py"
    click Element "https://github.com/mtehabsim/screenpilot/blob/main/core/element.py"
    click Sequence "https://github.com/mtehabsim/screenpilot/blob/main/core/action_sequence.py"
    click _ "https://github.com/mtehabsim/screenpilot/tree/main/core/"

    %% Styles
    classDef external fill:#A8FFB0,stroke:#333,stroke-width:1px
    classDef service fill:#ACE7FF,stroke:#333,stroke-width:1px
    classDef core fill:#FFCA8E,stroke:#333,stroke-width:1px
    classDef infra fill:#D3D3D3,stroke:#333,stroke-width:1px
```
