How mtehabsim/screenpilot works
Project type and purpose
• ScreenPilot is a Python-based “MCP server” (a micro-service) that exposes a suite of GUI automation tools (screen capture, mouse, keyboard, etc.) to a large-language-model client (e.g., Claude AI).
• It is not a web app or compiler: it’s a standalone automation service that integrates with an LLM via the MCP (Model-Control-Protocol) interface.
File structure
• Top-level entries:
• main.py ← service entrypoint, starts the MCP server
• config.py ← configuration loader for MCP registration
• requirements.txt
• core/ ← the heart of the toolkit
• utils.py ← shared helpers
• core/ contains:
• base_tool.py ← abstract base for all tools
• screen_capture.py ← screenshot and image‐analysis APIs
• mouse.py ← move/click methods
• keyboard.py ← typing and hotkey methods
• scroll.py ← scrolling control
• element.py ← element detection / waiting
• action_sequence.py ← orchestration of multiple tool calls
README for additional insights
• Describes how ScreenPilot plugs into the Claude AI desktop via a JSON config.
• Lists all available tools (screen, mouse, keyboard, scroll, element detection, sequences).