# midunthangavel/myuniverse architecture

> Architecture diagram of the GitHub repository midunthangavel/myuniverse, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-10-02). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/midunthangavel/myuniverse
- Repository: https://github.com/midunthangavel/myuniverse

## Overview

Synapse is a phone-native personal AI agent with a web cockpit, backend planning and memory services, and Android device automation. The primary workflow is a user request entering through the cockpit or backend API, being planned by the agent, then executed against a phone and checked against the resulting screen state. The README also describes voice, proactive routines, multimodal perception, and memory. Some links below follow explicit README or SOURCE INDEX references; unsampled subsystems are included where documented, but their internal wiring is not inferred.

Main entry points: [`backend/app/agent.py`](https://github.com/midunthangavel/myuniverse/blob/main/backend/app/agent.py) (Agent Orchestrator), [`src/main.js`](https://github.com/midunthangavel/myuniverse/blob/main/src/main.js) (Dashboard UI), [`backend/app/main.py`](https://github.com/midunthangavel/myuniverse/blob/main/backend/app/main.py) (API & WebSockets), [`backend/app/device/controller_factory.py`](https://github.com/midunthangavel/myuniverse/blob/main/backend/app/device/controller_factory.py) (Controller Factory)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_cockpit["Web Cockpit"]
  node_dashboard["Dashboard UI<br/>[main.js]"]
  node_cockpit_agent["Agent Interaction<br/>[agent.js]"]
  node_audio["Voice Interface<br/>[audio.js]"]
  node_avatar["Animated Avatar<br/>[character.js]"]
  node_phone_mirror["Phone Mirror<br/>[phone.js]"]
  node_memory_ui["Memory Explorer<br/>[memory.js]"]
end

subgraph group_backend["Agent Backend"]
  node_api["API &amp; WebSockets<br/>[main.py]"]
end

subgraph group_intelligence["Reasoning &amp; Memory"]
  node_agent["Agent Orchestrator<br/>[agent.py]"]
  node_fast["Fast Classifier<br/>[fast_classifier.py]"]
  node_llm["LLM Providers<br/>[llm_factory.py]"]
  node_vision["Vision Perception<br/>[vision.py]"]
  node_vlm["Local VLM<br/>[vlm_perception.py]"]
  node_prompts["Prompt Manager<br/>[prompt_manager.py]"]
  node_guardrails["Guardrails<br/>[guardrails.py]"]
  node_proactive["Proactive Routines<br/>[proactive.py]"]
  node_memory["Memory Services<br/>[memory.py]"]
  node_app_memory["App Trajectories<br/>[app_memory.py]"]
  node_vector_store[("Vector Memory<br/>[vector_memory.py]")]
  node_tools["Tool Registry<br/>[tool_registry.py]"]
  node_progress["Task Progressor<br/>[progressor.py]"]
  node_reflect["Action Reflector<br/>[reflector.py]"]
end

subgraph group_device["Phone Automation"]
  node_action["Action Engine<br/>[action_engine.py]"]
  node_state["State Engine<br/>[state_engine.py]"]
  node_grounding["UI Grounding"]
  node_verify["Outcome Verification"]
  node_parser["UI Parser<br/>[ui_parser.py]"]
  node_factory["Controller Factory"]
  node_android_controller["ADB Controller"]
  node_ios_controller["iOS Controller<br/>[ios_controller.py]"]
  node_sim_controller["Simulator Controller<br/>[sim_controller.py]"]
end

subgraph group_android["Android Client"]
  node_adb["ADB Bridge<br/>[adb_bridge.py]"]
  node_accessibility["Accessibility Service"]
  node_ws_client["WebSocket Client"]
end

node_user(("User"))

node_user -->|"uses"| node_dashboard
node_dashboard -->|"dispatches to"| node_cockpit_agent
node_dashboard -->|"wires"| node_audio
node_dashboard -->|"renders"| node_avatar
node_dashboard -->|"wires"| node_phone_mirror
node_dashboard -->|"wires"| node_memory_ui
node_cockpit_agent -.->|"requests"| node_api
node_api -->|"invokes"| node_agent
node_api -->|"uses"| node_proactive
node_api -->|"serves"| node_app_memory
node_api -->|"routes device calls"| node_factory
node_api -->|"configures"| node_llm
node_agent -->|"plans with"| node_llm
node_agent -->|"uses"| node_vision
node_agent -->|"uses"| node_prompts
node_agent -->|"checks"| node_guardrails
node_agent -->|"dispatches tools"| node_tools
node_agent -->|"uses"| node_fast
node_agent -->|"recalls"| node_memory
node_agent -->|"uses"| node_vector_store
node_agent -->|"tracks tasks"| node_progress
node_agent -->|"reflects with"| node_reflect
node_app_memory -->|"stores trajectories"| node_vector_store
node_vision -.->|"may use"| node_vlm
node_action -->|"perceives state"| node_state
node_action -->|"resolves targets"| node_grounding
node_action -->|"verifies actions"| node_verify
node_action -->|"gets controller"| node_factory
node_state -->|"uses"| node_parser
node_state -->|"reads hierarchy"| node_android_controller
node_verify -->|"compares states"| node_state
node_factory -->|"selects"| node_android_controller
node_factory -->|"selects"| node_ios_controller
node_factory -->|"selects"| node_sim_controller
node_android_controller -.->|"controls device via"| node_adb
node_ws_client -->|"uses"| node_accessibility

click node_dashboard "https://github.com/midunthangavel/myuniverse/blob/main/src/main.js"
click node_cockpit_agent "https://github.com/midunthangavel/myuniverse/blob/main/src/agent.js"
click node_audio "https://github.com/midunthangavel/myuniverse/blob/main/src/audio.js"
click node_avatar "https://github.com/midunthangavel/myuniverse/blob/main/src/character.js"
click node_phone_mirror "https://github.com/midunthangavel/myuniverse/blob/main/src/phone.js"
click node_memory_ui "https://github.com/midunthangavel/myuniverse/blob/main/src/memory.js"
click node_api "https://github.com/midunthangavel/myuniverse/blob/main/backend/app/main.py"
click node_agent "https://github.com/midunthangavel/myuniverse/blob/main/backend/app/agent.py"
click node_fast "https://github.com/midunthangavel/myuniverse/blob/main/backend/app/fast_classifier.py"
click node_llm "https://github.com/midunthangavel/myuniverse/blob/main/backend/app/llm_factory.py"
click node_vision "https://github.com/midunthangavel/myuniverse/blob/main/backend/app/vision.py"
click node_vlm "https://github.com/midunthangavel/myuniverse/blob/main/backend/app/vlm_perception.py"
click node_prompts "https://github.com/midunthangavel/myuniverse/blob/main/backend/app/prompt_manager.py"
click node_guardrails "https://github.com/midunthangavel/myuniverse/blob/main/backend/app/guardrails.py"
click node_proactive "https://github.com/midunthangavel/myuniverse/blob/main/backend/app/proactive.py"
click node_memory "https://github.com/midunthangavel/myuniverse/blob/main/backend/app/memory.py"
click node_app_memory "https://github.com/midunthangavel/myuniverse/blob/main/backend/app/app_memory.py"
click node_vector_store "https://github.com/midunthangavel/myuniverse/blob/main/backend/app/vector_memory.py"
click node_tools "https://github.com/midunthangavel/myuniverse/blob/main/backend/app/tool_registry.py"
click node_progress "https://github.com/midunthangavel/myuniverse/blob/main/backend/app/progressor.py"
click node_reflect "https://github.com/midunthangavel/myuniverse/blob/main/backend/app/reflector.py"
click node_action "https://github.com/midunthangavel/myuniverse/blob/main/backend/app/device/action_engine.py"
click node_state "https://github.com/midunthangavel/myuniverse/blob/main/backend/app/device/state_engine.py"
click node_grounding "https://github.com/midunthangavel/myuniverse/blob/main/backend/app/device/grounding_engine.py"
click node_verify "https://github.com/midunthangavel/myuniverse/blob/main/backend/app/device/verification_engine.py"
click node_parser "https://github.com/midunthangavel/myuniverse/blob/main/backend/app/device/ui_parser.py"
click node_factory "https://github.com/midunthangavel/myuniverse/blob/main/backend/app/device/controller_factory.py"
click node_android_controller "https://github.com/midunthangavel/myuniverse/blob/main/backend/app/device/android_controller.py"
click node_ios_controller "https://github.com/midunthangavel/myuniverse/blob/main/backend/app/device/ios_controller.py"
click node_sim_controller "https://github.com/midunthangavel/myuniverse/blob/main/backend/app/device/sim_controller.py"
click node_adb "https://github.com/midunthangavel/myuniverse/blob/main/android/bridge/adb_bridge.py"
click node_accessibility "https://github.com/midunthangavel/myuniverse/blob/main/android/app/src/main/java/com/synapse/ai/accessibility/SynapseAccessibilityService.kt"
click node_ws_client "https://github.com/midunthangavel/myuniverse/blob/main/android/app/src/main/java/com/synapse/ai/network/SynapseWebSocketClient.kt"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_dashboard,node_cockpit_agent,node_audio,node_avatar,node_phone_mirror,node_memory_ui,node_user toneBlue
class node_api toneAmber
class node_agent,node_fast,node_llm,node_vision,node_vlm,node_prompts,node_guardrails,node_proactive,node_memory,node_app_memory,node_vector_store,node_tools,node_progress,node_reflect toneMint
class node_action,node_state,node_grounding,node_verify,node_parser,node_factory,node_android_controller,node_ios_controller,node_sim_controller toneRose
class node_adb,node_accessibility,node_ws_client toneIndigo
```

## Components

### Web Cockpit

- [Dashboard UI](https://github.com/midunthangavel/myuniverse/blob/main/src/main.js): `src/main.js`
- [Agent Interaction](https://github.com/midunthangavel/myuniverse/blob/main/src/agent.js): `src/agent.js`
- [Voice Interface](https://github.com/midunthangavel/myuniverse/blob/main/src/audio.js): `src/audio.js`
- [Animated Avatar](https://github.com/midunthangavel/myuniverse/blob/main/src/character.js): `src/character.js`
- [Phone Mirror](https://github.com/midunthangavel/myuniverse/blob/main/src/phone.js): `src/phone.js`
- [Memory Explorer](https://github.com/midunthangavel/myuniverse/blob/main/src/memory.js): `src/memory.js`

### Agent Backend

- [API & WebSockets](https://github.com/midunthangavel/myuniverse/blob/main/backend/app/main.py): `backend/app/main.py`

### Reasoning & Memory

- [Agent Orchestrator](https://github.com/midunthangavel/myuniverse/blob/main/backend/app/agent.py): `backend/app/agent.py`
- [Fast Classifier](https://github.com/midunthangavel/myuniverse/blob/main/backend/app/fast_classifier.py): `backend/app/fast_classifier.py`
- [LLM Providers](https://github.com/midunthangavel/myuniverse/blob/main/backend/app/llm_factory.py): `backend/app/llm_factory.py`
- [Vision Perception](https://github.com/midunthangavel/myuniverse/blob/main/backend/app/vision.py): `backend/app/vision.py`
- [Local VLM](https://github.com/midunthangavel/myuniverse/blob/main/backend/app/vlm_perception.py): `backend/app/vlm_perception.py`
- [Prompt Manager](https://github.com/midunthangavel/myuniverse/blob/main/backend/app/prompt_manager.py): `backend/app/prompt_manager.py`
- [Guardrails](https://github.com/midunthangavel/myuniverse/blob/main/backend/app/guardrails.py): `backend/app/guardrails.py`
- [Proactive Routines](https://github.com/midunthangavel/myuniverse/blob/main/backend/app/proactive.py): `backend/app/proactive.py`
- [Memory Services](https://github.com/midunthangavel/myuniverse/blob/main/backend/app/memory.py): `backend/app/memory.py`
- [App Trajectories](https://github.com/midunthangavel/myuniverse/blob/main/backend/app/app_memory.py): `backend/app/app_memory.py`
- [Vector Memory](https://github.com/midunthangavel/myuniverse/blob/main/backend/app/vector_memory.py): `backend/app/vector_memory.py`
- [Tool Registry](https://github.com/midunthangavel/myuniverse/blob/main/backend/app/tool_registry.py): `backend/app/tool_registry.py`
- [Task Progressor](https://github.com/midunthangavel/myuniverse/blob/main/backend/app/progressor.py): `backend/app/progressor.py`
- [Action Reflector](https://github.com/midunthangavel/myuniverse/blob/main/backend/app/reflector.py): `backend/app/reflector.py`

### Phone Automation

- [Action Engine](https://github.com/midunthangavel/myuniverse/blob/main/backend/app/device/action_engine.py): `backend/app/device/action_engine.py`
- [State Engine](https://github.com/midunthangavel/myuniverse/blob/main/backend/app/device/state_engine.py): `backend/app/device/state_engine.py`
- [UI Grounding](https://github.com/midunthangavel/myuniverse/blob/main/backend/app/device/grounding_engine.py): `backend/app/device/grounding_engine.py`
- [Outcome Verification](https://github.com/midunthangavel/myuniverse/blob/main/backend/app/device/verification_engine.py): `backend/app/device/verification_engine.py`
- [UI Parser](https://github.com/midunthangavel/myuniverse/blob/main/backend/app/device/ui_parser.py): `backend/app/device/ui_parser.py`
- [Controller Factory](https://github.com/midunthangavel/myuniverse/blob/main/backend/app/device/controller_factory.py): `backend/app/device/controller_factory.py`
- [ADB Controller](https://github.com/midunthangavel/myuniverse/blob/main/backend/app/device/android_controller.py): `backend/app/device/android_controller.py`
- [iOS Controller](https://github.com/midunthangavel/myuniverse/blob/main/backend/app/device/ios_controller.py): `backend/app/device/ios_controller.py`
- [Simulator Controller](https://github.com/midunthangavel/myuniverse/blob/main/backend/app/device/sim_controller.py): `backend/app/device/sim_controller.py`

### Android Client

- [ADB Bridge](https://github.com/midunthangavel/myuniverse/blob/main/android/bridge/adb_bridge.py): `android/bridge/adb_bridge.py`
- [Accessibility Service](https://github.com/midunthangavel/myuniverse/blob/main/android/app/src/main/java/com/synapse/ai/accessibility/SynapseAccessibilityService.kt): `android/app/src/main/java/com/synapse/ai/accessibility/SynapseAccessibilityService.kt`
- [WebSocket Client](https://github.com/midunthangavel/myuniverse/blob/main/android/app/src/main/java/com/synapse/ai/network/SynapseWebSocketClient.kt): `android/app/src/main/java/com/synapse/ai/network/SynapseWebSocketClient.kt`

### Other components

- **User**

## Connections

- User → Dashboard UI: uses
- Dashboard UI → Agent Interaction: dispatches to (evidence: `src/main.js`)
- Dashboard UI → Voice Interface: wires (evidence: `src/main.js`)
- Dashboard UI → Animated Avatar: renders (evidence: `src/main.js`)
- Dashboard UI → Phone Mirror: wires (evidence: `src/main.js`)
- Dashboard UI → Memory Explorer: wires (evidence: `src/main.js`)
- Agent Interaction → API & WebSockets: requests
- API & WebSockets → Agent Orchestrator: invokes (evidence: `backend/app/main.py`)
- API & WebSockets → Proactive Routines: uses (evidence: `backend/app/main.py`)
- API & WebSockets → App Trajectories: serves (evidence: `backend/app/main.py`)
- API & WebSockets → Controller Factory: routes device calls (evidence: `backend/app/main.py`)
- API & WebSockets → LLM Providers: configures (evidence: `backend/app/main.py`)
- Agent Orchestrator → LLM Providers: plans with (evidence: `backend/app/agent.py`)
- Agent Orchestrator → Vision Perception: uses (evidence: `backend/app/agent.py`)
- Agent Orchestrator → Prompt Manager: uses (evidence: `backend/app/agent.py`)
- Agent Orchestrator → Guardrails: checks (evidence: `backend/app/agent.py`)
- Agent Orchestrator → Tool Registry: dispatches tools (evidence: `backend/app/agent.py`)
- Agent Orchestrator → Fast Classifier: uses (evidence: `backend/app/agent.py`)
- Agent Orchestrator → Memory Services: recalls (evidence: `backend/app/agent.py`)
- Agent Orchestrator → Vector Memory: uses (evidence: `backend/app/agent.py`)
- Agent Orchestrator → Task Progressor: tracks tasks (evidence: `backend/app/agent.py`)
- Agent Orchestrator → Action Reflector: reflects with (evidence: `backend/app/agent.py`)
- App Trajectories → Vector Memory: stores trajectories (evidence: `backend/app/app_memory.py`)
- Vision Perception → Local VLM: may use
- Action Engine → State Engine: perceives state (evidence: `backend/app/device/action_engine.py`)
- Action Engine → UI Grounding: resolves targets (evidence: `backend/app/device/action_engine.py`)
- Action Engine → Outcome Verification: verifies actions (evidence: `backend/app/device/action_engine.py`)
- Action Engine → Controller Factory: gets controller (evidence: `backend/app/device/action_engine.py`)
- State Engine → UI Parser: uses (evidence: `backend/app/device/state_engine.py`)
- State Engine → ADB Controller: reads hierarchy (evidence: `backend/app/device/state_engine.py`)
- Outcome Verification → State Engine: compares states (evidence: `backend/app/device/verification_engine.py`)
- Controller Factory → ADB Controller: selects (evidence: `backend/app/device/controller_factory.py`)
- Controller Factory → iOS Controller: selects (evidence: `backend/app/device/controller_factory.py`)
- Controller Factory → Simulator Controller: selects (evidence: `backend/app/device/controller_factory.py`)
- ADB Controller → ADB Bridge: controls device via
- WebSocket Client → Accessibility Service: uses (evidence: `android/app/src/main/java/com/synapse/ai/network/SynapseWebSocketClient.kt`)
