# lalalic/neox architecture

> Architecture diagram of the GitHub repository lalalic/neox, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-10-03). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/lalalic/neox
- Repository: https://github.com/lalalic/neox

## Overview

Neox is an iPhone media MCP server for desktop coding agents, with a companion Mac gateway called NeoY. The main workflow is: an agent connects to the phone over the LAN, searches or analyzes its photo library, then exports media for download. Separate user-facing paths support Siri handoff, guided Capture Tours, and asynchronous Vlog submissions. NeoY provides the Mac-side authenticated gateway and native capabilities, and federates a bundled MCP runtime. Several Mac-side links are documented capabilities; unsampled wiring is not inferred.

Main entry points: [`mac/NeoY/Sources/NeoY/MCPServer.swift`](https://github.com/lalalic/neox/blob/main/mac/NeoY/Sources/NeoY/MCPServer.swift) (NeoY MCP gateway), [`Neox/Server/ServerController.swift`](https://github.com/lalalic/neox/blob/main/Neox/Server/ServerController.swift) (Server lifecycle), [`Neox/Intents/RunAgentIntent.swift`](https://github.com/lalalic/neox/blob/main/Neox/Intents/RunAgentIntent.swift) (Run Agent intent), [`Neox/AgentKit/AppAgentToolProvider.swift`](https://github.com/lalalic/neox/blob/main/Neox/AgentKit/AppAgentToolProvider.swift) (UI pilot tools)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_phone["iPhone MCP service"]
  node_phone_mcp["Phone MCP server<br/>[MCPServer.swift]"]
  node_server_controller["Server lifecycle"]
  node_phone_status["Status screen<br/>[StatusScreen.swift]"]
end

subgraph group_media["Media and capture"]
  node_media_tools["Media tools<br/>[MediaTools.swift]"]
  node_vision_tools["Vision tools<br/>[VisionTools.swift]"]
  node_vision_index[("Persistent vision index<br/>[VisionIndex.swift]")]
  node_capture_tour["Capture Tour<br/>[CaptureTour.swift]"]
  node_tour_view["Tour runner UI"]
  node_vlog_submission["Vlog submission"]
end

subgraph group_agent["Agent interaction"]
  node_run_intent["Run Agent intent"]
  node_agent_handoff["Handoff message<br/>[AgentHandoff.swift]"]
  node_agent_bridge["Bridge discovery<br/>[AgentBridge.swift]"]
  node_pilot_tools["UI pilot tools"]
  node_accessibility_scan["Accessibility scanner"]
  node_interaction_engine["Interaction engine"]
  node_demo_tools["Demo tools"]
end

subgraph group_mac["NeoY Mac host"]
  node_neoy_host["NeoY capability host<br/>[NeoYApp.swift]"]
  node_neoy_mcp["NeoY MCP gateway<br/>[MCPServer.swift]"]
  node_phone_client["Phone client"]
  node_phone_tools["Phone tools"]
  node_federation["MCP federation"]
end

subgraph group_runtime["Bundled runtime"]
  node_bundled_runtime["Bundled MCP runtime<br/>[mcp-gateway.mjs]"]
  node_chat_adapter["ChatGPT adapter"]
  node_chat_lifecycle["Chat lifecycle<br/>[lifecycle.mjs]"]
end

node_desktop_agent(("Desktop agent"))
node_icloud_folder[("iCloud Drive folder")]

node_desktop_agent -->|"calls tools"| node_phone_mcp
node_server_controller -->|"starts server"| node_phone_mcp
node_server_controller -->|"registers tools"| node_media_tools
node_server_controller -->|"registers tools"| node_vision_tools
node_media_tools -->|"reads analysis"| node_vision_index
node_desktop_agent -->|"requests analysis"| node_vision_tools
node_desktop_agent -->|"starts tour"| node_capture_tour
node_capture_tour -->|"uses file cache"| node_media_tools
node_phone_status -->|"opens runner"| node_tour_view
node_phone_status -->|"opens submission"| node_vlog_submission
node_vlog_submission -->|"writes submission"| node_icloud_folder
node_icloud_folder -->|"signals ready"| node_desktop_agent
node_run_intent -->|"starts server"| node_server_controller
node_run_intent -->|"composes message"| node_agent_handoff
node_run_intent -->|"posts handoff"| node_agent_bridge
node_pilot_tools -->|"scans interface"| node_accessibility_scan
node_pilot_tools -->|"dispatches actions"| node_interaction_engine
node_pilot_tools -->|"provides demo commands"| node_demo_tools
node_neoy_host -.->|"hosts gateway"| node_neoy_mcp
node_neoy_mcp -->|"federates runtime"| node_bundled_runtime
node_neoy_mcp -.->|"exposes phone tools"| node_phone_tools
node_phone_tools -.->|"uses transport"| node_phone_client
node_phone_client -->|"connects over HTTP"| node_phone_mcp
node_neoy_mcp -->|"uses adapter"| node_chat_adapter
node_neoy_mcp -->|"uses lifecycle"| node_chat_lifecycle
node_neoy_mcp -.->|"federates services"| node_federation

click node_phone_mcp "https://github.com/lalalic/neox/blob/main/Neox/Server/MCPServer.swift"
click node_server_controller "https://github.com/lalalic/neox/blob/main/Neox/Server/ServerController.swift"
click node_phone_status "https://github.com/lalalic/neox/blob/main/Neox/App/StatusScreen.swift"
click node_media_tools "https://github.com/lalalic/neox/blob/main/Neox/Server/MediaTools.swift"
click node_vision_tools "https://github.com/lalalic/neox/blob/main/Neox/Server/VisionTools.swift"
click node_vision_index "https://github.com/lalalic/neox/blob/main/Neox/Server/VisionIndex.swift"
click node_capture_tour "https://github.com/lalalic/neox/blob/main/Neox/Server/CaptureTour.swift"
click node_tour_view "https://github.com/lalalic/neox/blob/main/Neox/App/CaptureTourView.swift"
click node_vlog_submission "https://github.com/lalalic/neox/blob/main/Neox/App/VlogSubmission.swift"
click node_run_intent "https://github.com/lalalic/neox/blob/main/Neox/Intents/RunAgentIntent.swift"
click node_agent_handoff "https://github.com/lalalic/neox/blob/main/Neox/Intents/AgentHandoff.swift"
click node_agent_bridge "https://github.com/lalalic/neox/blob/main/Neox/Intents/AgentBridge.swift"
click node_pilot_tools "https://github.com/lalalic/neox/blob/main/Neox/AgentKit/AppAgentToolProvider.swift"
click node_accessibility_scan "https://github.com/lalalic/neox/blob/main/Neox/AgentKit/AccessibilityScanner.swift"
click node_interaction_engine "https://github.com/lalalic/neox/blob/main/Neox/AgentKit/InteractionEngine.swift"
click node_demo_tools "https://github.com/lalalic/neox/blob/main/Neox/AgentKit/DemoToolProvider.swift"
click node_neoy_host "https://github.com/lalalic/neox/blob/main/mac/NeoY/Sources/NeoY/NeoYApp.swift"
click node_neoy_mcp "https://github.com/lalalic/neox/blob/main/mac/NeoY/Sources/NeoY/MCPServer.swift"
click node_phone_client "https://github.com/lalalic/neox/blob/main/mac/NeoY/Sources/NeoY/NeoXPhoneClient.swift"
click node_phone_tools "https://github.com/lalalic/neox/blob/main/mac/NeoY/Sources/NeoY/NeoXPhoneTools.swift"
click node_federation "https://github.com/lalalic/neox/blob/main/mac/NeoY/Sources/NeoY/MCPFederation.swift"
click node_bundled_runtime "https://github.com/lalalic/neox/blob/main/mac/NeoY/Runtime/src/core/mcp-gateway.mjs"
click node_chat_adapter "https://github.com/lalalic/neox/blob/main/mac/NeoY/Runtime/src/xchat/chatgpt-responses-adapter.mjs"
click node_chat_lifecycle "https://github.com/lalalic/neox/blob/main/mac/NeoY/Runtime/src/xchat/lifecycle.mjs"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_phone_mcp,node_server_controller,node_phone_status toneBlue
class node_media_tools,node_vision_tools,node_vision_index,node_capture_tour,node_tour_view,node_vlog_submission,node_icloud_folder toneAmber
class node_run_intent,node_agent_handoff,node_agent_bridge,node_pilot_tools,node_accessibility_scan,node_interaction_engine,node_demo_tools toneMint
class node_neoy_host,node_neoy_mcp,node_phone_client,node_phone_tools,node_federation toneRose
class node_bundled_runtime,node_chat_adapter,node_chat_lifecycle,node_desktop_agent toneIndigo
```

## Components

### iPhone MCP service

- [Phone MCP server](https://github.com/lalalic/neox/blob/main/Neox/Server/MCPServer.swift): `Neox/Server/MCPServer.swift`
- [Server lifecycle](https://github.com/lalalic/neox/blob/main/Neox/Server/ServerController.swift): `Neox/Server/ServerController.swift`
- [Status screen](https://github.com/lalalic/neox/blob/main/Neox/App/StatusScreen.swift): `Neox/App/StatusScreen.swift`

### Media and capture

- [Media tools](https://github.com/lalalic/neox/blob/main/Neox/Server/MediaTools.swift): `Neox/Server/MediaTools.swift`
- [Vision tools](https://github.com/lalalic/neox/blob/main/Neox/Server/VisionTools.swift): `Neox/Server/VisionTools.swift`
- [Persistent vision index](https://github.com/lalalic/neox/blob/main/Neox/Server/VisionIndex.swift): `Neox/Server/VisionIndex.swift`
- [Capture Tour](https://github.com/lalalic/neox/blob/main/Neox/Server/CaptureTour.swift): `Neox/Server/CaptureTour.swift`
- [Tour runner UI](https://github.com/lalalic/neox/blob/main/Neox/App/CaptureTourView.swift): `Neox/App/CaptureTourView.swift`
- [Vlog submission](https://github.com/lalalic/neox/blob/main/Neox/App/VlogSubmission.swift): `Neox/App/VlogSubmission.swift`

### Agent interaction

- [Run Agent intent](https://github.com/lalalic/neox/blob/main/Neox/Intents/RunAgentIntent.swift): `Neox/Intents/RunAgentIntent.swift`
- [Handoff message](https://github.com/lalalic/neox/blob/main/Neox/Intents/AgentHandoff.swift): `Neox/Intents/AgentHandoff.swift`
- [Bridge discovery](https://github.com/lalalic/neox/blob/main/Neox/Intents/AgentBridge.swift): `Neox/Intents/AgentBridge.swift`
- [UI pilot tools](https://github.com/lalalic/neox/blob/main/Neox/AgentKit/AppAgentToolProvider.swift): `Neox/AgentKit/AppAgentToolProvider.swift`
- [Accessibility scanner](https://github.com/lalalic/neox/blob/main/Neox/AgentKit/AccessibilityScanner.swift): `Neox/AgentKit/AccessibilityScanner.swift`
- [Interaction engine](https://github.com/lalalic/neox/blob/main/Neox/AgentKit/InteractionEngine.swift): `Neox/AgentKit/InteractionEngine.swift`
- [Demo tools](https://github.com/lalalic/neox/blob/main/Neox/AgentKit/DemoToolProvider.swift): `Neox/AgentKit/DemoToolProvider.swift`

### NeoY Mac host

- [NeoY capability host](https://github.com/lalalic/neox/blob/main/mac/NeoY/Sources/NeoY/NeoYApp.swift): `mac/NeoY/Sources/NeoY/NeoYApp.swift`
- [NeoY MCP gateway](https://github.com/lalalic/neox/blob/main/mac/NeoY/Sources/NeoY/MCPServer.swift): `mac/NeoY/Sources/NeoY/MCPServer.swift`
- [Phone client](https://github.com/lalalic/neox/blob/main/mac/NeoY/Sources/NeoY/NeoXPhoneClient.swift): `mac/NeoY/Sources/NeoY/NeoXPhoneClient.swift`
- [Phone tools](https://github.com/lalalic/neox/blob/main/mac/NeoY/Sources/NeoY/NeoXPhoneTools.swift): `mac/NeoY/Sources/NeoY/NeoXPhoneTools.swift`
- [MCP federation](https://github.com/lalalic/neox/blob/main/mac/NeoY/Sources/NeoY/MCPFederation.swift): `mac/NeoY/Sources/NeoY/MCPFederation.swift`

### Bundled runtime

- [Bundled MCP runtime](https://github.com/lalalic/neox/blob/main/mac/NeoY/Runtime/src/core/mcp-gateway.mjs): `mac/NeoY/Runtime/src/core/mcp-gateway.mjs`
- [ChatGPT adapter](https://github.com/lalalic/neox/blob/main/mac/NeoY/Runtime/src/xchat/chatgpt-responses-adapter.mjs): `mac/NeoY/Runtime/src/xchat/chatgpt-responses-adapter.mjs`
- [Chat lifecycle](https://github.com/lalalic/neox/blob/main/mac/NeoY/Runtime/src/xchat/lifecycle.mjs): `mac/NeoY/Runtime/src/xchat/lifecycle.mjs`

### Other components

- **Desktop agent**
- **iCloud Drive folder**

## Connections

- Desktop agent → Phone MCP server: calls tools (evidence: `README.md`)
- Server lifecycle → Phone MCP server: starts server (evidence: `Neox/Server/ServerController.swift`)
- Server lifecycle → Media tools: registers tools (evidence: `Neox/Server/ServerController.swift`)
- Server lifecycle → Vision tools: registers tools (evidence: `Neox/Server/ServerController.swift`)
- Media tools → Persistent vision index: reads analysis (evidence: `Neox/Server/MediaTools.swift`)
- Desktop agent → Vision tools: requests analysis (evidence: `README.md`)
- Desktop agent → Capture Tour: starts tour (evidence: `README.md`)
- Capture Tour → Media tools: uses file cache (evidence: `Neox/Server/CaptureTour.swift`)
- Status screen → Tour runner UI: opens runner (evidence: `Neox/App/StatusScreen.swift`)
- Status screen → Vlog submission: opens submission (evidence: `Neox/App/StatusScreen.swift`)
- Vlog submission → iCloud Drive folder: writes submission (evidence: `README.md`)
- iCloud Drive folder → Desktop agent: signals ready (evidence: `README.md`)
- Run Agent intent → Server lifecycle: starts server (evidence: `Neox/Intents/RunAgentIntent.swift`)
- Run Agent intent → Handoff message: composes message (evidence: `Neox/Intents/RunAgentIntent.swift`)
- Run Agent intent → Bridge discovery: posts handoff (evidence: `Neox/Intents/RunAgentIntent.swift`)
- UI pilot tools → Accessibility scanner: scans interface (evidence: `Neox/AgentKit/AppAgentToolProvider.swift`)
- UI pilot tools → Interaction engine: dispatches actions (evidence: `Neox/AgentKit/AppAgentToolProvider.swift`)
- UI pilot tools → Demo tools: provides demo commands (evidence: `Neox/AgentKit/AppAgentToolProvider.swift`)
- NeoY capability host → NeoY MCP gateway: hosts gateway
- NeoY MCP gateway → Bundled MCP runtime: federates runtime (evidence: `README.md`)
- NeoY MCP gateway → Phone tools: exposes phone tools
- Phone tools → Phone client: uses transport
- Phone client → Phone MCP server: connects over HTTP (evidence: `README.md`)
- NeoY MCP gateway → ChatGPT adapter: uses adapter
- NeoY MCP gateway → Chat lifecycle: uses lifecycle
- NeoY MCP gateway → MCP federation: federates services
