How chrisroyse/synapse works
Synapse is a local Windows automation runtime that exposes desktop perception and control to AI models through MCP. The principal loop is an agent request, MCP tool dispatch, observation of the desktop or browser, a guarded action, and a returned result; durable agent, audit, and timeline state supports ongoing work. The tree and README support these major capability areas, but the sampled excerpts do not expose most tool-handler wiring, so subsystem links below reflect documented behavior rather than claims of inspected call sites.
Main entry points: mod.rs Tool handlers · observe.rs Observe and find · server.rs MCP server · lib.rs Durable state
Components 21
MCP Gateway
- MCP server
crates/synapse-mcp/src/server.rs - MCP transports
crates/synapse-mcp/src/http/transport.rs - Tool handlers
crates/synapse-mcp/src/server/m1_tools/mod.rs
Perception
- Screen capture
crates/synapse-capture/src/lib.rs - UI Automation
crates/synapse-a11y/src/lib.rs - Observe and find
crates/synapse-perception/src/observe.rs - OCR and text
crates/synapse-perception/src/ocr.rs - Browser tools
crates/synapse-mcp/src/server/browser_facades.rs - Audio sensing
crates/synapse-audio/src/lib.rs
Desktop Control
- Action tools
crates/synapse-mcp/src/m2.rs - Input execution
crates/synapse-action/src/lib.rs
Agent Continuity
- Reflex runtime
crates/synapse-reflex/src/runtime.rs - Durable state
crates/synapse-storage/src/lib.rs - Agent operations
crates/synapse-mcp/src/server/agent_query.rs - Routines and plans
crates/synapse-mcp/src/m3/routines.rs
Operator UI
- Dashboard
dashboard/src/app.tsx - Tray companion
crates/synapse-overlay/src/main.rs
Other components
- AI agent
- Chrome extension
extensions/synapse-chrome-debugger/service_worker.js - Windows desktop
- Synapse daemon
crates/synapse-mcp/src/main.rs
Connections 21
- AI agent → MCP transports: sends MCP requests
- MCP transports → MCP server: delivers requests
- MCP server → Tool handlers: dispatches tools
- Tool handlers → Observe and find: requests observation
- Observe and find → Screen capture: captures frames
- Observe and find → UI Automation: reads UI tree
- Observe and find → OCR and text: extracts text
- Tool handlers → Browser tools: dispatches browser tools
- Browser tools → Chrome extension: sends browser commands
- Tool handlers → Action tools: dispatches actions
- Action tools → Input execution: executes input
- Input execution → Windows desktop: controls desktop
- Screen capture → Windows desktop: captures screen
- UI Automation → Windows desktop: reads controls
- Tool handlers → Audio sensing: requests audio
- MCP server → Reflex runtime: uses runtime
- Reflex runtime → Durable state: persists state
- Agent operations → Durable state: reads agent records
- Routines and plans → Durable state: uses durable state
- Dashboard → Synapse daemon: queries and controls
- Tray companion → Synapse daemon: queries daemon