How mediar-ai/terminator works
Project type and purpose
• From the README and file tree, terminator is an AI-first, cross-platform UI-automation library + server written in Rust, with language-specific SDKs (TypeScript and Python) and various example clients. Its purpose is to let upstream clients (web apps, Python scripts, Rust programs) drive native GUI apps on Windows/macOS via a server using OS-level accessibility APIs.
File structure
• Top-level Rust core
• Cargo.toml, src/lib.rs, src/element.rs, src/locator.rs, src/selector.rs, src/utils.rs, src/errors.rs, src/tests.rs
• src/platforms/{windows.rs, macos.rs, linux.rs, tree_search.rs, mod.rs}
• Server binaries & examples
• setup_windows.ps1 → installs prebuilt server.exe
• examples/server.rs, example .exe under server_release/ (built via build-release.yml)
• Language SDKs (clients)
• ts-sdk/ (TypeScript client wrapper, package.json, src/index.ts)
• python-sdk/desktop_use (Python client wrapper, client.py, models.py, exceptions.py)
• MCP CLI tool
• mcp/ (TypeScript CLI “terminator cli” for “Vibe Work using MCP”)
• Web and script examples
• examples/hello-world (Next.js app calling ts-sdk through API routes)
• examples/pdf-to-form (Node example showing AI + terminator integration)
• examples/*.rs, examples/*.py (standalone Rust/Python demos)
README for additional insights
• Describes a client–server architecture: you build/run a server (server.exe or macOS binary), then you run client code (Rust example, TS SDK, Python SDK) that connects to it.
• Mentions OS accessibility APIs (uiautomation-rs on Windows, AppKit on macOS) as underlying dependencies.
a. Core Terminator Engine (Rust lib & server)
• Platform Modules: windows.rs, macos.rs, linux.rs, tree_search.rs
• Core Logic: element.rs, selector.rs, locator.rs, utils.rs
• Server Layer: server.rs (accepts incoming RPC/HTTP/WebSocket requests)
b. Communication Layer
• Protocol: JSON-RPC or HTTP API over localhost (port X)
• Serialization: arguments & return values (element handles, metadata)
c. Language SDKs
• TypeScript SDK (ts-sdk/src/index.ts): exposes a Playwright-like API, wraps HTTP calls to server.
• Python SDK (python-sdk/desktop_use/client.py): similar wrapper in Python.
d. Client Applications
• Web Example: examples/hello-world (Next.js + API routes + ts-sdk)
• Node Example: examples/pdf-to-form (Node.js script + ts-sdk)
• Rust/Python Examples: examples/simple.rs, examples/client_example.py
e. External Dependencies & Debugging Tools
• OS Accessibility APIs (uiautomation-rs on Windows, NSAccessibility on macOS)
• Dev tools: Accessibility Insights, FlaUInspect
Architectural patterns & design principles
• Modular Platform Abstraction: each OS lives in its own module under src/platforms.
• Server–Client Separation: heavy native automation logic runs in a single server process; SDKs remain thin clients.
• Language-agnostic API: JSON-RPC/HTTP boundaries let you support any client language.
• Extensibility Points: new platform implementations, new SDKs, new client examples.
Diagramming details
• Client → SDK → Server API → Core Engine → OS Accessibility API
• Core Engine → Platform Module
• Green for Rust native components (core library & server)
• Blue for SDKs (ts-sdk, python-sdk)
• Orange for clients (examples/**)
• Gray for external systems (OS APIs, debugging tools)
• Ports/Protocols: annotate “HTTP/JSON-RPC over localhost:port” on the boundary between SDK and server.