# secondly-com/openphone architecture

> Architecture diagram of the GitHub repository secondly-com/openphone, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-06-26). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/secondly-com/openphone
- Repository: https://github.com/secondly-com/openphone

## Overview

1. **What this repo is**
   OpenPhone is not a standalone app repo; it is the OpenPhone-owned control plane for an Android-based phone OS. It contains the Android overlay, privileged assistant app, framework patches, build/release tooling, schemas, and a reference model-broker service. The actual Android source tree is not vendored here; this repo is applied on top of LineageOS 23.2 / Android 16.

2. **Top-level architecture**
   The system is split into four main layers: OpenPhone UI/surface, the privileged assistant runtime, OS-level services/patches, and external model infrastructure. The user interacts with the OpenPhone surface; the assistant turns that into mediated actions; framework services expose device state and execution capabilities; model services provide reasoning, transcription, and realtime voice.

3. **Android integration boundary**
   The `overlay/` directory holds the packaged OpenPhone app, resources, and prebuilts that get copied into the Android tree. The `patches/` directory modifies upstream AOSP/LineageOS components such as `frameworks_base`, `SystemUI`, `Settings`, `system_sepolicy`, and build tooling. This is where OpenPhone becomes a system product rather than an app layered on top of stock Android.

4. **Privileged assistant runtime**
   `overlay/packages/apps/OpenPhoneAssistant/` is the core app. It is a privileged system component with Compose-based UI (`MainActivity`, `AssistantComposeHost`, chat and advanced screens), an AIDL interface, and both Java/Kotlin services. It owns the orchestrator, model adapters, tool execution, policy checks, memory, commitments, watchers, jobs, and audit/export logic.

5. **User-facing surfaces**
   The assistant is exposed through several OS entry points rather than only a launcher icon: an always-available system surface, a Quick Settings tile, a boot receiver, a trigger receiver, a notification listener, and an accessibility service. There is also a pointer overlay/controller for mediated interaction on top of other apps. These surfaces are how the assistant can stay present, be re-entered, and act while the user remains in the phone context.

6. **Action execution and OS mediation**
   Actions are not executed directly against apps from the model layer. They pass through OpenPhone’s action registry/catalog, task registry, framework tool executor, and OS-owned services introduced by platform patches. The framework side includes services such as `openphone_agent`, `openphone_context`, and `openphone_assistant_data`, plus mediated input, screenshot, UI-tree, clipboard, and URL/opening flows. This is the main trust boundary between the assistant and the device.

7. **Model and voice layer**
   The model abstraction is centralized under `src/.../model/`. It supports multiple backends: Gemini Live voice sessions, OpenAI Realtime voice sessions, OpenAI Responses agent mode, speech transcription, and a local heuristic adapter for fallback or validation. A separate reference Python service in `services/model-broker/` brokers model connectivity and deployment, so the assistant can talk to external providers without hardcoding provider-specific access into the app.

8. **Context, memory, and proactive automation**
   The runtime keeps state across turns through several stores: context events/index, memory records, commitments, jobs, watchers, and trajectories. The orchestrator and job scheduler enable background runs, heartbeats, and exact scheduled workflows. Watchers monitor phone state such as notifications, calls, app foreground state, visible screen state, battery, connectivity, and commitments so the assistant can react proactively, not just conversationally.

9. **Policy, approvals, and auditability**
   OpenPhone’s safety model is a first-class subsystem. The policy engine evaluates capabilities and risk, confirmations gate sensitive actions, and audit logging records decisions and evidence. The app also exports trajectories and audit evidence, and the repo includes schemas for action requests/results, audit logs, trajectory events, screen context, and model tools. This makes the system inspectable and reviewable after the fact.

10. **Device and platform scope**
   The first supported target is Pixel 9a (`tegu`), with generic ARM64 builds used mainly for graph validation. Device notes, bootchain docs, and support matrices live under `docs/devices/` and `docs/TEGU_BOOTCHAIN.md`. The repo also contains Android product manifests for the OpenPhone target and scripts for flashing, device checks, and hardware validation.

11. **Build, sync, and release pipeline**
   `scripts/` drives the lifecycle: syncing the Android tree, applying patches, building products, pushing the assistant APK, flashing devices, and running validation/evals. GitHub Actions cover CI, eval, and release workflows. Release artifacts, OTA feeds, and manifests are generated and validated by dedicated scripts and JSON schemas, which indicates a controlled, reproducible release process rather than ad hoc builds.

12. **What to draw in an architecture diagram**
   The key diagram boundaries are: user inputs → OpenPhone surface; surface → privileged assistant; assistant ↔ model broker/external models; assistant ↔ OS services/framework patches; OS services ↔ device context/apps; and assistant/policy ↔ audit/trajectory exports. A separate box should represent the repo’s build/release/tooling plane, because it assembles and validates the system but is not part of runtime execution.

Main entry points: [`overlay/packages/apps/OpenPhoneAssistant/src/org/openphone/assistant`](https://github.com/secondly-com/openphone/tree/main/overlay/packages/apps/OpenPhoneAssistant/src/org/openphone/assistant) (Assistant runtime), [`scripts`](https://github.com/secondly-com/openphone/tree/main/scripts) (Scripts), [`overlay/packages/apps/OpenPhoneAssistant/src/org/openphone/assistant/agent`](https://github.com/secondly-com/openphone/tree/main/overlay/packages/apps/OpenPhoneAssistant/src/org/openphone/assistant/agent) (Tool executor), [`schemas`](https://github.com/secondly-com/openphone/tree/main/schemas) (Schemas)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_ui["OpenPhone surface"]
  node_openphone_app["OpenPhone app<br/>android overlay"]
  node_assistant_ui["Assistant UI<br/>compose surface"]
  node_entry_surfaces["Entry points<br/>system surfaces"]
end

subgraph group_runtime["Assistant runtime"]
  node_assistant_runtime{{"Assistant runtime<br/>privileged app"}}
  node_orchestrator["Orchestrator<br/>control loop"]
  node_policy_engine["Policy engine<br/>safety gate"]
  node_state_stores[("State stores<br/>persistence")]
  node_actions_catalog["Action catalog<br/>capability registry"]
  node_tool_executor["Tool executor<br/>device mediation"]
  node_model_layer["Model layer<br/>multi-backend adapter"]
  node_assistant_audit["Audit export<br/>evidence trail"]
end

subgraph group_platform["Platform patches"]
  node_framework_services{{"Framework services<br/>aosp patches"}}
  node_mediation_hooks["Mediation hooks<br/>os primitives"]
  node_sepolicy["SELinux policy<br/>system policy"]
  node_settings_systemui["Settings/SystemUI<br/>ui patches"]
end

subgraph group_infra["Model infra"]
  node_model_broker["Model broker<br/>python service"]
  node_external_models["External models<br/>provider backends"]
end

subgraph group_tooling["Build & release"]
  node_scripts["Scripts<br/>automation"]
  node_schemas["Schemas<br/>json contracts"]
  node_ci_release["CI & release<br/>github workflows<br/>[workflows]"]
  node_manifests["Manifests<br/>product config<br/>[openphone.xml]"]
end

node_openphone_app -->|"contains"| node_assistant_ui
node_openphone_app -->|"exposes"| node_entry_surfaces
node_assistant_ui -->|"drives"| node_assistant_runtime
node_entry_surfaces -->|"enters"| node_assistant_runtime
node_assistant_runtime -->|"coordinates"| node_orchestrator
node_assistant_runtime -->|"checks"| node_policy_engine
node_assistant_runtime -->|"persists"| node_state_stores
node_assistant_runtime -->|"selects"| node_actions_catalog
node_assistant_runtime -->|"executes"| node_tool_executor
node_assistant_runtime -->|"reasons with"| node_model_layer
node_assistant_runtime -->|"records"| node_assistant_audit
node_tool_executor -->|"calls"| node_framework_services
node_tool_executor -->|"uses"| node_mediation_hooks
node_framework_services -->|"constrained by"| node_sepolicy
node_settings_systemui -->|"adds"| node_entry_surfaces
node_model_layer -->|"connects"| node_external_models
node_model_broker -->|"brokers"| node_external_models
node_scripts -->|"syncs"| node_manifests
node_scripts -->|"builds"| node_assistant_runtime
node_scripts -->|"applies"| node_framework_services
node_scripts -->|"deploys"| node_model_broker
node_scripts -->|"validates"| node_schemas
node_ci_release -->|"orchestrates"| node_scripts
node_schemas -->|"defines"| node_assistant_audit
node_schemas -->|"defines"| node_state_stores

click node_openphone_app "https://github.com/secondly-com/openphone/tree/main/overlay/packages/apps/OpenPhoneAssistant"
click node_assistant_ui "https://github.com/secondly-com/openphone/tree/main/overlay/packages/apps/OpenPhoneAssistant/src/org/openphone/assistant"
click node_entry_surfaces "https://github.com/secondly-com/openphone/tree/main/overlay/packages/apps/OpenPhoneAssistant/src/org/openphone/assistant"
click node_assistant_runtime "https://github.com/secondly-com/openphone/tree/main/overlay/packages/apps/OpenPhoneAssistant/src/org/openphone/assistant"
click node_orchestrator "https://github.com/secondly-com/openphone/tree/main/overlay/packages/apps/OpenPhoneAssistant/src/org/openphone/assistant/orchestrator"
click node_policy_engine "https://github.com/secondly-com/openphone/tree/main/overlay/packages/apps/OpenPhoneAssistant/src/org/openphone/assistant/policy"
click node_state_stores "https://github.com/secondly-com/openphone/tree/main/overlay/packages/apps/OpenPhoneAssistant/src/org/openphone/assistant"
click node_actions_catalog "https://github.com/secondly-com/openphone/tree/main/overlay/packages/apps/OpenPhoneAssistant/src/org/openphone/assistant/actions"
click node_tool_executor "https://github.com/secondly-com/openphone/tree/main/overlay/packages/apps/OpenPhoneAssistant/src/org/openphone/assistant/agent"
click node_model_layer "https://github.com/secondly-com/openphone/tree/main/overlay/packages/apps/OpenPhoneAssistant/src/org/openphone/assistant/model"
click node_assistant_audit "https://github.com/secondly-com/openphone/tree/main/overlay/packages/apps/OpenPhoneAssistant/src/org/openphone/assistant/agent"
click node_framework_services "https://github.com/secondly-com/openphone/tree/main/patches/frameworks_base"
click node_mediation_hooks "https://github.com/secondly-com/openphone/tree/main/patches/frameworks_base"
click node_sepolicy "https://github.com/secondly-com/openphone/tree/main/patches/system_sepolicy"
click node_settings_systemui "https://github.com/secondly-com/openphone/tree/main/patches/packages_apps_Settings"
click node_model_broker "https://github.com/secondly-com/openphone/blob/main/services/model-broker/openphone_model_broker.py"
click node_external_models "https://github.com/secondly-com/openphone/tree/main/overlay/packages/apps/OpenPhoneAssistant/src/org/openphone/assistant/model"
click node_scripts "https://github.com/secondly-com/openphone/tree/main/scripts"
click node_schemas "https://github.com/secondly-com/openphone/tree/main/schemas"
click node_ci_release "https://github.com/secondly-com/openphone/blob/main/.github/workflows"
click node_manifests "https://github.com/secondly-com/openphone/blob/main/manifests/openphone.xml"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_openphone_app,node_assistant_ui,node_entry_surfaces toneBlue
class node_assistant_runtime,node_orchestrator,node_policy_engine,node_state_stores,node_actions_catalog,node_tool_executor,node_model_layer,node_assistant_audit toneAmber
class node_framework_services,node_mediation_hooks,node_sepolicy,node_settings_systemui toneMint
class node_model_broker,node_external_models toneRose
class node_scripts,node_schemas,node_ci_release,node_manifests toneIndigo
```

## Components

### OpenPhone surface

Packaged Android app and user-facing entry points.

- [OpenPhone app](https://github.com/secondly-com/openphone/tree/main/overlay/packages/apps/OpenPhoneAssistant): android overlay · Packaged assistant app and resources injected into the Android tree. · `overlay/packages/apps/OpenPhoneAssistant`
- [Assistant UI](https://github.com/secondly-com/openphone/tree/main/overlay/packages/apps/OpenPhoneAssistant/src/org/openphone/assistant): compose surface · Launcher/activity, chat, advanced screens, and shared Compose host. · `overlay/packages/apps/OpenPhoneAssistant/src/org/openphone/assistant`
- [Entry points](https://github.com/secondly-com/openphone/tree/main/overlay/packages/apps/OpenPhoneAssistant/src/org/openphone/assistant): system surfaces · QS tile, boot/trigger receivers, accessibility, notification, and pointer overlay entry paths. · `overlay/packages/apps/OpenPhoneAssistant/src/org/openphone/assistant`

### Assistant runtime

Privileged orchestration, policy, state, and model integration.

- [Assistant runtime](https://github.com/secondly-com/openphone/tree/main/overlay/packages/apps/OpenPhoneAssistant/src/org/openphone/assistant): privileged app · Core system assistant component coordinating model calls, tools, state, and safety. · `overlay/packages/apps/OpenPhoneAssistant/src/org/openphone/assistant`
- [Orchestrator](https://github.com/secondly-com/openphone/tree/main/overlay/packages/apps/OpenPhoneAssistant/src/org/openphone/assistant/orchestrator): control loop · Selects operating mode and drives agent decisions and execution. · `overlay/packages/apps/OpenPhoneAssistant/src/org/openphone/assistant/orchestrator`
- [Policy engine](https://github.com/secondly-com/openphone/tree/main/overlay/packages/apps/OpenPhoneAssistant/src/org/openphone/assistant/policy): safety gate · Evaluates capability risk, confirmations, and audit decisions before sensitive actions. · `overlay/packages/apps/OpenPhoneAssistant/src/org/openphone/assistant/policy`
- [State stores](https://github.com/secondly-com/openphone/tree/main/overlay/packages/apps/OpenPhoneAssistant/src/org/openphone/assistant): persistence · Context, memory, commitments, jobs, watchers, and trajectory records. · `overlay/packages/apps/OpenPhoneAssistant/src/org/openphone/assistant`
- [Action catalog](https://github.com/secondly-com/openphone/tree/main/overlay/packages/apps/OpenPhoneAssistant/src/org/openphone/assistant/actions): capability registry · Action registry and tool catalog defining available assistant capabilities. · `overlay/packages/apps/OpenPhoneAssistant/src/org/openphone/assistant/actions`
- [Tool executor](https://github.com/secondly-com/openphone/tree/main/overlay/packages/apps/OpenPhoneAssistant/src/org/openphone/assistant/agent): device mediation · Framework tool executor and screen understanding used to execute mediated actions. · `overlay/packages/apps/OpenPhoneAssistant/src/org/openphone/assistant/agent`
- [Model layer](https://github.com/secondly-com/openphone/tree/main/overlay/packages/apps/OpenPhoneAssistant/src/org/openphone/assistant/model): multi-backend adapter · Abstracts voice, transcription, and agent backends across providers and local fallback. · `overlay/packages/apps/OpenPhoneAssistant/src/org/openphone/assistant/model`
- [Audit export](https://github.com/secondly-com/openphone/tree/main/overlay/packages/apps/OpenPhoneAssistant/src/org/openphone/assistant/agent): evidence trail · Trajectory recorder and audit evidence exporter for reviewable action history. · `overlay/packages/apps/OpenPhoneAssistant/src/org/openphone/assistant/agent`

### Platform patches

Android framework and system integration boundary.

- [Framework services](https://github.com/secondly-com/openphone/tree/main/patches/frameworks_base): aosp patches · Platform-added system services such as agent manager, context, and assistant data. · `patches/frameworks_base`
- [Mediation hooks](https://github.com/secondly-com/openphone/tree/main/patches/frameworks_base): os primitives · Foreground context, screenshot, UI tree, clipboard, URL, pointer, and input mediation. · `patches/frameworks_base`
- [SELinux policy](https://github.com/secondly-com/openphone/tree/main/patches/system_sepolicy): system policy · Labels and confines assistant-related services and system data access. · `patches/system_sepolicy`
- [Settings/SystemUI](https://github.com/secondly-com/openphone/tree/main/patches/packages_apps_Settings): ui patches · System surfaces for agent tile, settings pages, and device/version integration. · `patches/packages_apps_Settings`

### Model infra

External model brokerage and deployment surface.

- [Model broker](https://github.com/secondly-com/openphone/blob/main/services/model-broker/openphone_model_broker.py): python service · Reference deployment service that brokers connectivity to external model providers. · `services/model-broker/openphone_model_broker.py`
- [External models](https://github.com/secondly-com/openphone/tree/main/overlay/packages/apps/OpenPhoneAssistant/src/org/openphone/assistant/model): provider backends · Gemini Live, OpenAI Realtime, Responses, transcription, and heuristic fallback. · `overlay/packages/apps/OpenPhoneAssistant/src/org/openphone/assistant/model`

### Build & release

Repo lifecycle, validation, and release automation.

- [Scripts](https://github.com/secondly-com/openphone/tree/main/scripts): automation · Sync, build, flash, validation, benchmark, release, and device tooling. · `scripts`
- [Schemas](https://github.com/secondly-com/openphone/tree/main/schemas): json contracts · Action, audit, trajectory, policy, model tool, OTA, and task data contracts. · `schemas`
- [CI & release](https://github.com/secondly-com/openphone/blob/main/.github/workflows): github workflows · Continuous integration, eval, and release workflows for reproducible delivery. · `.github/workflows`
- [Manifests](https://github.com/secondly-com/openphone/blob/main/manifests/openphone.xml): product config · Repo manifest used to sync the Android tree and assemble OpenPhone targets. · `manifests/openphone.xml`

## Connections

- OpenPhone app → Assistant UI: contains · The packaged app includes the Compose UI surface and resources.
- OpenPhone app → Entry points: exposes · The app publishes system entry points and background receivers.
- Assistant UI → Assistant runtime: drives · User interactions enter the privileged runtime through UI state and actions.
- Entry points → Assistant runtime: enters · QS, receivers, accessibility, and overlays can start or re-enter the assistant.
- Assistant runtime → Orchestrator: coordinates · The runtime delegates turn and task control to the orchestrator.
- Assistant runtime → Policy engine: checks · Sensitive or risky actions are gated by policy and confirmations.
- Assistant runtime → State stores: persists · Runtime state, jobs, memory, and watchers are stored across turns.
- Assistant runtime → Action catalog: selects · The runtime resolves available capabilities from the action registry.
- Assistant runtime → Tool executor: executes · Approved actions flow into mediated device execution.
- Assistant runtime → Model layer: reasons with · The runtime invokes voice, agent, and transcription backends.
- Assistant runtime → Audit export: records · Decisions and evidence are exported for later review.
- Tool executor → Framework services: calls · Execution uses OS-owned services added by platform patches.
- Tool executor → Mediation hooks: uses · Mediated execution relies on screenshots, UI trees, input, clipboard, and URL flows.
- Framework services → SELinux policy: constrained by · New system services and data paths are labeled and confined by policy.
- Settings/SystemUI → Entry points: adds · System UI and Settings patches add assistant-facing surfaces.
- Model layer → External models: connects · Adapters reach external providers or local fallback logic.
- Model broker → External models: brokers · The reference broker mediates deployment connectivity to providers.
- Scripts → Manifests: syncs · Automation uses manifests to assemble the Android tree.
- Scripts → Assistant runtime: builds · Build and push scripts package and deploy the assistant app.
- Scripts → Framework services: applies · Patch application and build steps produce the platform integration layer.
- Scripts → Model broker: deploys · Tooling supports broker setup, smoke tests, and secret rotation.
- Scripts → Schemas: validates · Release and eval scripts validate generated artifacts against schemas.
- CI & release → Scripts: orchestrates · GitHub Actions run the repository automation in CI, eval, and release flows.
- Schemas → Audit export: defines · Schemas describe exported audit, trajectory, and action records.
- Schemas → State stores: defines · Schemas also capture jobs, tasks, policy, and screen context contracts.