How core-mate/opengui works
OpenGUI is an Android GUI-agent framework with companion integrations for DeepSeek Harness and WorkBuddy, plus a full-stack task backend. The user’s main workflow is to request a phone operation, let an agent observe the device and choose actions, execute them, and return results. The graph also shows the Android client’s task and accessibility path. Several backend and client subsystems are represented by their enclosing directories because their implementations were not sampled; unsupported internal links are omitted.
Main entry points: phone-controller.ts DSH phone control · GestureService.kt Gesture service · index.ts DeepSeek Harness · service.ts WorkBuddy connector
Components 21
Agent integrations
- DeepSeek Harness
deepseek-harness-plugin/src/index.ts - WorkBuddy connector
workbuddy-plugin/src/service.ts
Phone operation
- DSH phone control
deepseek-harness-plugin/src/phone-controller.ts - WorkBuddy phone control
workbuddy-plugin/src/phone-controller.ts - DSH device commands
deepseek-harness-plugin/src/adb.ts
Android client
- Android app UI
client/feature_promotor/src/main/java/com/coremate/opengui/feature/promotor/ui - Task API client
client/core_network/src/main/java/com/coremate/opengui/network/api/ApiService.kt - Accessibility actions
client/core_accessibility/src/main/java/com/coremate/opengui/accessibility/ActionExecutor.kt - Gesture service
client/core_accessibility/src/main/java/com/coremate/opengui/accessibility/GestureService.kt - Local log storage
client/core_common/src/main/java/com/coremate/opengui/common/log/LogManager.kt
Task backend
- Task endpoints
server/apps/backend/src/modules/task/task.controller.ts - Task execution services
server/apps/backend/src/modules/task - Creator agent
server/apps/backend/src/modules/creator-agent/creator-agent.controller.ts
Automation runtime
- Automation service
client/automation/src/main/java/com/coremate/opengui/automation/AMServiceManager.kt - Automation core
client/automation/src/main/java/com/coremate/opengui/automation/base/AMCore.kt - Task implementations
client/automation/src/main/java/com/coremate/opengui/automation/biz/tasks - Remote dispatch
client/core_network/src/main/java/com/coremate/opengui/network/websocket/StandbyForegroundService.kt - Automation events
client/core_common_jvm/src/main/java/com/coremate/opengui/common_jvm/event/AutomationEvent.kt
Other components
- User
- Authorized Android phone
- Visual model
Connections 17
- User → DeepSeek Harness: requests task
README.md - User → WorkBuddy connector: requests task
README.md - DeepSeek Harness → DSH phone control: operates phone
README.md - WorkBuddy connector → WorkBuddy phone control: operates phone
README.md - DSH phone control → DSH device commands: builds commands
deepseek-harness-plugin/src/phone-controller.ts - DSH phone control → Authorized Android phone: captures and acts
README.md - WorkBuddy phone control → Authorized Android phone: captures and acts
README.md - Visual model → Authorized Android phone: guides operation
README.md - Android app UI → Task API client: requests tasks
- Task API client → Task endpoints: calls endpoints
- Task endpoints → Task execution services: delegates execution
server/apps/backend/src/modules/task/task.controller.ts - Creator agent → Task execution services: coordinates tasks
- Automation service → Automation core: queues work
client/automation/src/main/java/com/coremate/opengui/automation/AMServiceManager.kt - Automation core → Task implementations: runs tasks
client/automation/src/main/java/com/coremate/opengui/automation/base/AMCore.kt - Remote dispatch → Automation events: publishes dispatch
client/core_network/src/main/java/com/coremate/opengui/network/websocket/StandbyForegroundService.kt - Gesture service → Automation events: publishes events
client/core_accessibility/src/main/java/com/coremate/opengui/accessibility/GestureService.kt - Gesture service → Local log storage: saves logs
client/core_accessibility/src/main/java/com/coremate/opengui/accessibility/GestureService.kt