# core-mate/opengui architecture

> Architecture diagram of the GitHub repository core-mate/opengui, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-10-02). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/core-mate/opengui
- Repository: https://github.com/core-mate/opengui

## Overview

OpenGUI is an Android GUI-agent framework with companion integrations for DeepSeek Harness and WorkBuddy, plus a full-stack task backend. The user’s main workflow is to request a phone operation, let an agent observe the device and choose actions, execute them, and return results. The graph also shows the Android client’s task and accessibility path. Several backend and client subsystems are represented by their enclosing directories because their implementations were not sampled; unsupported internal links are omitted.

Main entry points: [`deepseek-harness-plugin/src/phone-controller.ts`](https://github.com/core-mate/opengui/blob/main/deepseek-harness-plugin/src/phone-controller.ts) (DSH phone control), [`client/core_accessibility/src/main/java/com/coremate/opengui/accessibility/GestureService.kt`](https://github.com/core-mate/opengui/blob/main/client/core_accessibility/src/main/java/com/coremate/opengui/accessibility/GestureService.kt) (Gesture service), [`deepseek-harness-plugin/src/index.ts`](https://github.com/core-mate/opengui/blob/main/deepseek-harness-plugin/src/index.ts) (DeepSeek Harness), [`workbuddy-plugin/src/service.ts`](https://github.com/core-mate/opengui/blob/main/workbuddy-plugin/src/service.ts) (WorkBuddy connector)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_integrations["Agent integrations"]
  node_dsh["DeepSeek Harness<br/>[index.ts]"]
  node_workbuddy["WorkBuddy connector<br/>[service.ts]"]
end

subgraph group_phone["Phone operation"]
  node_dsh_phone["DSH phone control"]
  node_wb_phone["WorkBuddy phone control"]
  node_dsh_adb["DSH device commands<br/>[adb.ts]"]
end

subgraph group_android["Android client"]
  node_app_ui["Android app UI"]
  node_task_api["Task API client<br/>[ApiService.kt]"]
  node_accessibility["Accessibility actions<br/>[ActionExecutor.kt]"]
  node_gesture_service["Gesture service<br/>[GestureService.kt]"]
  node_log_store[("Local log storage<br/>[LogManager.kt]")]
end

subgraph group_backend["Task backend"]
  node_backend_api["Task endpoints<br/>[task.controller.ts]"]
  node_task_services["Task execution services"]
  node_creator_agent["Creator agent"]
end

subgraph group_automation["Automation runtime"]
  node_automation_service["Automation service"]
  node_automation_core["Automation core<br/>[AMCore.kt]"]
  node_automation_tasks["Task implementations"]
  node_remote_dispatch["Remote dispatch"]
  node_event_bus["Automation events<br/>[AutomationEvent.kt]"]
end

node_user(("User"))
node_phone_device(("Authorized Android phone"))
node_visual_model(("Visual model"))

node_user -->|"requests task"| node_dsh
node_user -->|"requests task"| node_workbuddy
node_dsh -->|"operates phone"| node_dsh_phone
node_workbuddy -->|"operates phone"| node_wb_phone
node_dsh_phone -->|"builds commands"| node_dsh_adb
node_dsh_phone -->|"captures and acts"| node_phone_device
node_wb_phone -->|"captures and acts"| node_phone_device
node_visual_model -->|"guides operation"| node_phone_device
node_app_ui -.->|"requests tasks"| node_task_api
node_task_api -.->|"calls endpoints"| node_backend_api
node_backend_api -->|"delegates execution"| node_task_services
node_creator_agent -.->|"coordinates tasks"| node_task_services
node_automation_service -.->|"queues work"| node_automation_core
node_automation_core -.->|"runs tasks"| node_automation_tasks
node_remote_dispatch -->|"publishes dispatch"| node_event_bus
node_gesture_service -->|"publishes events"| node_event_bus
node_gesture_service -->|"saves logs"| node_log_store

click node_dsh "https://github.com/core-mate/opengui/blob/main/deepseek-harness-plugin/src/index.ts"
click node_workbuddy "https://github.com/core-mate/opengui/blob/main/workbuddy-plugin/src/service.ts"
click node_dsh_phone "https://github.com/core-mate/opengui/blob/main/deepseek-harness-plugin/src/phone-controller.ts"
click node_wb_phone "https://github.com/core-mate/opengui/blob/main/workbuddy-plugin/src/phone-controller.ts"
click node_dsh_adb "https://github.com/core-mate/opengui/blob/main/deepseek-harness-plugin/src/adb.ts"
click node_app_ui "https://github.com/core-mate/opengui/tree/main/client/feature_promotor/src/main/java/com/coremate/opengui/feature/promotor/ui"
click node_task_api "https://github.com/core-mate/opengui/blob/main/client/core_network/src/main/java/com/coremate/opengui/network/api/ApiService.kt"
click node_accessibility "https://github.com/core-mate/opengui/blob/main/client/core_accessibility/src/main/java/com/coremate/opengui/accessibility/ActionExecutor.kt"
click node_gesture_service "https://github.com/core-mate/opengui/blob/main/client/core_accessibility/src/main/java/com/coremate/opengui/accessibility/GestureService.kt"
click node_backend_api "https://github.com/core-mate/opengui/blob/main/server/apps/backend/src/modules/task/task.controller.ts"
click node_task_services "https://github.com/core-mate/opengui/tree/main/server/apps/backend/src/modules/task"
click node_creator_agent "https://github.com/core-mate/opengui/blob/main/server/apps/backend/src/modules/creator-agent/creator-agent.controller.ts"
click node_automation_service "https://github.com/core-mate/opengui/blob/main/client/automation/src/main/java/com/coremate/opengui/automation/AMServiceManager.kt"
click node_automation_core "https://github.com/core-mate/opengui/blob/main/client/automation/src/main/java/com/coremate/opengui/automation/base/AMCore.kt"
click node_automation_tasks "https://github.com/core-mate/opengui/tree/main/client/automation/src/main/java/com/coremate/opengui/automation/biz/tasks"
click node_remote_dispatch "https://github.com/core-mate/opengui/blob/main/client/core_network/src/main/java/com/coremate/opengui/network/websocket/StandbyForegroundService.kt"
click node_event_bus "https://github.com/core-mate/opengui/blob/main/client/core_common_jvm/src/main/java/com/coremate/opengui/common_jvm/event/AutomationEvent.kt"
click node_log_store "https://github.com/core-mate/opengui/blob/main/client/core_common/src/main/java/com/coremate/opengui/common/log/LogManager.kt"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_dsh,node_workbuddy,node_user toneBlue
class node_dsh_phone,node_wb_phone,node_dsh_adb toneAmber
class node_app_ui,node_task_api,node_accessibility,node_gesture_service,node_log_store toneMint
class node_backend_api,node_task_services,node_creator_agent toneRose
class node_automation_service,node_automation_core,node_automation_tasks,node_remote_dispatch,node_event_bus,node_phone_device,node_visual_model toneIndigo
```

## Components

### Agent integrations

- [DeepSeek Harness](https://github.com/core-mate/opengui/blob/main/deepseek-harness-plugin/src/index.ts): `deepseek-harness-plugin/src/index.ts`
- [WorkBuddy connector](https://github.com/core-mate/opengui/blob/main/workbuddy-plugin/src/service.ts): `workbuddy-plugin/src/service.ts`

### Phone operation

- [DSH phone control](https://github.com/core-mate/opengui/blob/main/deepseek-harness-plugin/src/phone-controller.ts): `deepseek-harness-plugin/src/phone-controller.ts`
- [WorkBuddy phone control](https://github.com/core-mate/opengui/blob/main/workbuddy-plugin/src/phone-controller.ts): `workbuddy-plugin/src/phone-controller.ts`
- [DSH device commands](https://github.com/core-mate/opengui/blob/main/deepseek-harness-plugin/src/adb.ts): `deepseek-harness-plugin/src/adb.ts`

### Android client

- [Android app UI](https://github.com/core-mate/opengui/tree/main/client/feature_promotor/src/main/java/com/coremate/opengui/feature/promotor/ui): `client/feature_promotor/src/main/java/com/coremate/opengui/feature/promotor/ui`
- [Task API client](https://github.com/core-mate/opengui/blob/main/client/core_network/src/main/java/com/coremate/opengui/network/api/ApiService.kt): `client/core_network/src/main/java/com/coremate/opengui/network/api/ApiService.kt`
- [Accessibility actions](https://github.com/core-mate/opengui/blob/main/client/core_accessibility/src/main/java/com/coremate/opengui/accessibility/ActionExecutor.kt): `client/core_accessibility/src/main/java/com/coremate/opengui/accessibility/ActionExecutor.kt`
- [Gesture service](https://github.com/core-mate/opengui/blob/main/client/core_accessibility/src/main/java/com/coremate/opengui/accessibility/GestureService.kt): `client/core_accessibility/src/main/java/com/coremate/opengui/accessibility/GestureService.kt`
- [Local log storage](https://github.com/core-mate/opengui/blob/main/client/core_common/src/main/java/com/coremate/opengui/common/log/LogManager.kt): `client/core_common/src/main/java/com/coremate/opengui/common/log/LogManager.kt`

### Task backend

- [Task endpoints](https://github.com/core-mate/opengui/blob/main/server/apps/backend/src/modules/task/task.controller.ts): `server/apps/backend/src/modules/task/task.controller.ts`
- [Task execution services](https://github.com/core-mate/opengui/tree/main/server/apps/backend/src/modules/task): `server/apps/backend/src/modules/task`
- [Creator agent](https://github.com/core-mate/opengui/blob/main/server/apps/backend/src/modules/creator-agent/creator-agent.controller.ts): `server/apps/backend/src/modules/creator-agent/creator-agent.controller.ts`

### Automation runtime

- [Automation service](https://github.com/core-mate/opengui/blob/main/client/automation/src/main/java/com/coremate/opengui/automation/AMServiceManager.kt): `client/automation/src/main/java/com/coremate/opengui/automation/AMServiceManager.kt`
- [Automation core](https://github.com/core-mate/opengui/blob/main/client/automation/src/main/java/com/coremate/opengui/automation/base/AMCore.kt): `client/automation/src/main/java/com/coremate/opengui/automation/base/AMCore.kt`
- [Task implementations](https://github.com/core-mate/opengui/tree/main/client/automation/src/main/java/com/coremate/opengui/automation/biz/tasks): `client/automation/src/main/java/com/coremate/opengui/automation/biz/tasks`
- [Remote dispatch](https://github.com/core-mate/opengui/blob/main/client/core_network/src/main/java/com/coremate/opengui/network/websocket/StandbyForegroundService.kt): `client/core_network/src/main/java/com/coremate/opengui/network/websocket/StandbyForegroundService.kt`
- [Automation events](https://github.com/core-mate/opengui/blob/main/client/core_common_jvm/src/main/java/com/coremate/opengui/common_jvm/event/AutomationEvent.kt): `client/core_common_jvm/src/main/java/com/coremate/opengui/common_jvm/event/AutomationEvent.kt`

### Other components

- **User**
- **Authorized Android phone**
- **Visual model**

## Connections

- User → DeepSeek Harness: requests task (evidence: `README.md`)
- User → WorkBuddy connector: requests task (evidence: `README.md`)
- DeepSeek Harness → DSH phone control: operates phone (evidence: `README.md`)
- WorkBuddy connector → WorkBuddy phone control: operates phone (evidence: `README.md`)
- DSH phone control → DSH device commands: builds commands (evidence: `deepseek-harness-plugin/src/phone-controller.ts`)
- DSH phone control → Authorized Android phone: captures and acts (evidence: `README.md`)
- WorkBuddy phone control → Authorized Android phone: captures and acts (evidence: `README.md`)
- Visual model → Authorized Android phone: guides operation (evidence: `README.md`)
- Android app UI → Task API client: requests tasks
- Task API client → Task endpoints: calls endpoints
- Task endpoints → Task execution services: delegates execution (evidence: `server/apps/backend/src/modules/task/task.controller.ts`)
- Creator agent → Task execution services: coordinates tasks
- Automation service → Automation core: queues work (evidence: `client/automation/src/main/java/com/coremate/opengui/automation/AMServiceManager.kt`)
- Automation core → Task implementations: runs tasks (evidence: `client/automation/src/main/java/com/coremate/opengui/automation/base/AMCore.kt`)
- Remote dispatch → Automation events: publishes dispatch (evidence: `client/core_network/src/main/java/com/coremate/opengui/network/websocket/StandbyForegroundService.kt`)
- Gesture service → Automation events: publishes events (evidence: `client/core_accessibility/src/main/java/com/coremate/opengui/accessibility/GestureService.kt`)
- Gesture service → Local log storage: saves logs (evidence: `client/core_accessibility/src/main/java/com/coremate/opengui/accessibility/GestureService.kt`)
