# andokivilaid/omnigent architecture

> Architecture diagram of the GitHub repository andokivilaid/omnigent, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-06-16). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/andokivilaid/omnigent
- Repository: https://github.com/andokivilaid/omnigent

## Overview

1. **This is a multi-surface AI orchestration platform, not a single app.**  
   Omnigent combines a Python backend/orchestrator, a React/TypeScript web UI, a terminal/CLI experience, desktop wrappers, and a set of agent harnesses that connect to external LLM providers and local tools.

2. **User-facing surfaces are split into three main entry points.**  
   - `omnigent` / `omni` CLI for terminal-first sessions and host registration  
   - `ap-web` for the browser UI  
   - Electron packaging under `ap-web/electron` for the macOS desktop app that wraps the same web UI and adds OS integrations

3. **The core control plane lives in `omnigent/server` and `omnigent/runner`.**  
   The server owns auth, sessions, hosts, policies, comments, permissions, and persistence-facing APIs.  
   The runner is the execution side: it hosts agent sessions, dispatches tools, manages approvals, and brokers runtime communication back to the server.

4. **Session state is central and persistent.**  
   Conversations, agents, files, comments, permissions, policies, hosts, and related metadata are stored through SQLAlchemy-backed stores in `omnigent/stores` with schema and migrations in `omnigent/db`.  
   The server exposes these resources through HTTP APIs and WebSocket/SSE streams so the web UI and CLI can observe live updates.

5. **Agent execution is abstracted behind harnesses and executors.**  
   `omnigent/inner` contains the concrete harness integrations for Claude, Codex, Cursor, Pi, OpenAI Agents SDK, Antigravity, Databricks, and other modes.  
   This layer is where the platform normalizes different agent runtimes into a common execution model.

6. **Policies are first-class and enforced in the runtime path.**  
   Policy definitions live under `omnigent/policies` and runtime enforcement under `omnigent/runtime/policies`.  
   They gate shell access, file writes, tool calls, routing, and spending decisions before actions are executed, so policy checks sit on the critical path between agent intent and tool execution.

7. **Tools are a separate subsystem from agents.**  
   `omnigent/tools` defines built-in tools, client-specified tools, MCP integration, and tool dispatch/validation.  
   Tools are invoked by the runner/runtime, can be local Python callables or external tool servers, and are surfaced back into conversations as structured events.

8. **Sandboxing and host execution are explicit boundaries.**  
   Local and cloud execution environments are handled in `omnigent/sandbox`, `omnigent/host`, and `omnigent/onboarding/sandboxes`.  
   The repo supports macOS seatbelt, Linux bubblewrap, and managed/cloud sandboxes such as Modal, Daytona, Islo, Cloud Workstations, Kubernetes, and Docker-based deployments.  
   This boundary isolates agent work from the user machine and from the server process.

9. **Model/provider selection is decoupled from orchestration.**  
   `omnigent/llms` contains provider adapters for Anthropic, OpenAI, Gemini, Bedrock, Vertex, Databricks, and gateway-style endpoints.  
   The onboarding flow and model catalog under `omnigent/onboarding` and `omnigent/model_catalog.py` choose credentials and defaults, while the runtime consumes a provider-neutral LLM client.

10. **The web app is a live session console, not just a chat UI.**  
   `ap-web/src` is organized around pages, shell panels, chat blocks, file viewers, terminal views, comments, approvals, and agent controls.  
   It consumes the server’s session streams to render conversation updates, sub-agent activity, terminal output, file diffs, and policy prompts in real time.

11. **Live collaboration is built into the architecture.**  
   The system supports shared sessions, co-driving, forks, and multi-user presence.  
   That shows up in server routes for presence/auth/session sharing, runner transports, and UI features for session switching, comments, and approvals.

12. **Transport is layered to support local, remote, and tunneled execution.**  
   The runner exposes transports over TCP, UDS, and WebSocket tunnels in `omnigent/runner/transports`, while the server has tunnel routes for hosts and runners.  
   This allows the same session model to work whether the agent is local on a laptop or running on a managed remote host.

13. **Deployment is multi-target and configuration-driven.**  
   The `deploy/` tree contains Docker, Kubernetes, Fly, Railway, Render, Modal, Hugging Face Spaces, and Daytona/CWSandbox setups.  
   The repo is designed so the same server/runner architecture can be deployed in several environments with different auth, database, and sandbox backends.

14. **SDKs and examples are secondary integration surfaces.**  
   `sdks/python-client` exposes the server API to external programs, while `sdks/ui` provides terminal-rendering helpers.  
   `examples/` and `tests/resources/` demonstrate how to define YAML agents, skills, sub-agents, tools, and policy stacks that plug into the same orchestration pipeline.

15. **A useful diagram should show four main zones and the arrows between them.**  
   - Client surfaces: CLI, web UI, Electron  
   - Control plane: server, auth, persistence, collaboration APIs  
   - Execution plane: runner, runtime, policies, tools, transports  
   - External dependencies: model providers, sandboxes, databases, OIDC/auth, MCP/tool servers  
   The key flows are user input → session/event stream → policy/runtime enforcement → harness/tool execution → persisted session updates → UI/CLI refresh.

Main entry points: [`omnigent/server/app.py`](https://github.com/andokivilaid/omnigent/blob/main/omnigent/server/app.py) (Server), [`omnigent/runner/app.py`](https://github.com/andokivilaid/omnigent/blob/main/omnigent/runner/app.py) (Runner), [`omnigent/runtime`](https://github.com/andokivilaid/omnigent/tree/main/omnigent/runtime) (Runtime), [`ap-web/src`](https://github.com/andokivilaid/omnigent/tree/main/ap-web/src) (Web UI)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_clients["Client surfaces"]
  node_cli["CLI<br/>terminal entry<br/>[cli.py]"]
  node_web["Web UI<br/>react console"]
  node_desktop["Desktop app<br/>electron wrapper"]
end

subgraph group_control["Control plane"]
  node_server["Server<br/>http/ws API<br/>[app.py]"]
  node_auth["Auth<br/>oidc/accounts<br/>[auth.py]"]
  node_sessions_api["Sessions API<br/>session routes<br/>[sessions.py]"]
  node_collab_api["Collaboration<br/>presence/comments<br/>[comments.py]"]
  node_host_api["Host API<br/>host/tunnel routes<br/>[hosts.py]"]
end

subgraph group_execution["Execution plane"]
  node_runner["Runner<br/>execution host<br/>[app.py]"]
  node_runtime["Runtime<br/>session engine"]
  node_policies{{"Policies<br/>enforcement"}}
  node_tools["Tools<br/>tool dispatch"]
  node_harnesses["Harnesses<br/>agent runtimes"]
  node_transports["Transports<br/>ws/tcp/uds"]
end

subgraph group_data["Persistence"]
  node_stores[("Stores")]
  node_db[("DB schema<br/>alembic models")]
end

subgraph group_deps["External systems"]
  node_llms["LLM adapters<br/>provider adapters"]
  node_sandboxes["Sandboxes<br/>execution isolation"]
  node_external_tools["External tools<br/>mcp/tool servers<br/>[mcp.py]"]
end

node_cli -->|"session API"| node_server
node_web -->|"live console"| node_server
node_desktop -->|"wraps UI"| node_web
node_server -->|"authenticate"| node_auth
node_server -->|"session routes"| node_sessions_api
node_server -->|"collaboration"| node_collab_api
node_server -->|"host registry"| node_host_api
node_server -->|"persist state"| node_stores
node_stores -->|"schema/migrations"| node_db
node_runner -->|"execute workflow"| node_runtime
node_runtime -->|"enforce"| node_policies
node_runtime -->|"invoke tools"| node_tools
node_runtime -->|"spawn agents"| node_harnesses
node_runner -->|"connect"| node_transports
node_runner -->|"stream events"| node_server
node_harnesses -->|"model calls"| node_llms
node_harnesses -->|"isolate work"| node_sandboxes
node_tools -->|"MCP/local"| node_external_tools
node_server -->|"tunnel/attach"| node_runner
node_web -->|"stream updates"| node_sessions_api
node_cli -->|"terminal session"| node_runner

click node_cli "https://github.com/andokivilaid/omnigent/blob/main/omnigent/cli.py"
click node_web "https://github.com/andokivilaid/omnigent/tree/main/ap-web/src"
click node_desktop "https://github.com/andokivilaid/omnigent/tree/main/ap-web/electron/src"
click node_server "https://github.com/andokivilaid/omnigent/blob/main/omnigent/server/app.py"
click node_auth "https://github.com/andokivilaid/omnigent/blob/main/omnigent/server/auth.py"
click node_sessions_api "https://github.com/andokivilaid/omnigent/blob/main/omnigent/server/routes/sessions.py"
click node_collab_api "https://github.com/andokivilaid/omnigent/blob/main/omnigent/server/routes/comments.py"
click node_host_api "https://github.com/andokivilaid/omnigent/blob/main/omnigent/server/routes/hosts.py"
click node_stores "https://github.com/andokivilaid/omnigent/tree/main/omnigent/stores"
click node_db "https://github.com/andokivilaid/omnigent/tree/main/omnigent/db"
click node_runner "https://github.com/andokivilaid/omnigent/blob/main/omnigent/runner/app.py"
click node_runtime "https://github.com/andokivilaid/omnigent/tree/main/omnigent/runtime"
click node_policies "https://github.com/andokivilaid/omnigent/tree/main/omnigent/runtime/policies"
click node_tools "https://github.com/andokivilaid/omnigent/tree/main/omnigent/tools"
click node_harnesses "https://github.com/andokivilaid/omnigent/tree/main/omnigent/inner"
click node_transports "https://github.com/andokivilaid/omnigent/tree/main/omnigent/runner/transports"
click node_llms "https://github.com/andokivilaid/omnigent/tree/main/omnigent/llms/adapters"
click node_sandboxes "https://github.com/andokivilaid/omnigent/tree/main/omnigent/sandbox"
click node_external_tools "https://github.com/andokivilaid/omnigent/blob/main/omnigent/tools/mcp.py"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_cli,node_web,node_desktop toneBlue
class node_server,node_auth,node_sessions_api,node_collab_api,node_host_api toneAmber
class node_runner,node_runtime,node_policies,node_tools,node_harnesses,node_transports toneMint
class node_stores,node_db toneRose
class node_llms,node_sandboxes,node_external_tools toneIndigo
```

## Components

### Client surfaces

Terminal, web, and desktop entry points for users.

- [CLI](https://github.com/andokivilaid/omnigent/blob/main/omnigent/cli.py): terminal entry · Terminal-first command surface for sessions, host registration, and REPL flows. · `omnigent/cli.py`
- [Web UI](https://github.com/andokivilaid/omnigent/tree/main/ap-web/src): react console · Browser-based live session console with chat, files, approvals, comments, and terminals. · `ap-web/src`
- [Desktop app](https://github.com/andokivilaid/omnigent/tree/main/ap-web/electron/src): electron wrapper · Electron packaging around the web UI with macOS integration. · `ap-web/electron/src`

### Control plane

Server APIs, auth, collaboration, and session persistence boundaries.

- [Server](https://github.com/andokivilaid/omnigent/blob/main/omnigent/server/app.py): http/ws API · Central control plane for auth, sessions, hosts, permissions, comments, and collaboration streams. · `omnigent/server/app.py`
- [Auth](https://github.com/andokivilaid/omnigent/blob/main/omnigent/server/auth.py): oidc/accounts · Login, account bootstrap, OIDC, and access control plumbing. · `omnigent/server/auth.py`
- [Sessions API](https://github.com/andokivilaid/omnigent/blob/main/omnigent/server/routes/sessions.py): session routes · Session CRUD, sharing, forks, state updates, and live event endpoints. · `omnigent/server/routes/sessions.py`
- [Collaboration](https://github.com/andokivilaid/omnigent/blob/main/omnigent/server/routes/comments.py): presence/comments · Presence, comments, approvals, and multi-user collaboration routes. · `omnigent/server/routes/comments.py`
- [Host API](https://github.com/andokivilaid/omnigent/blob/main/omnigent/server/routes/hosts.py): host/tunnel routes · Host registry, tunnel endpoints, worktree launch, and remote runner connections. · `omnigent/server/routes/hosts.py`

### Execution plane

Runner, runtime, policies, tools, harnesses, and transports.

- [Runner](https://github.com/andokivilaid/omnigent/blob/main/omnigent/runner/app.py): execution host · Runtime execution side that hosts agent sessions, dispatches tools, and brokers communication. · `omnigent/runner/app.py`
- [Runtime](https://github.com/andokivilaid/omnigent/tree/main/omnigent/runtime): session engine · Workflow, compaction, prompts, streaming, approvals, and runtime orchestration logic. · `omnigent/runtime`
- [Policies](https://github.com/andokivilaid/omnigent/tree/main/omnigent/runtime/policies): enforcement · Policy definitions and runtime enforcement for shell, file, tool, routing, and spending decisions. · `omnigent/runtime/policies`
- [Tools](https://github.com/andokivilaid/omnigent/tree/main/omnigent/tools): tool dispatch · Built-in, client-specified, local, and MCP-backed tools with validation and dispatch. · `omnigent/tools`
- [Harnesses](https://github.com/andokivilaid/omnigent/tree/main/omnigent/inner): agent runtimes · Adapters that normalize Claude, Codex, Cursor, Pi, OpenAI Agents SDK, Antigravity, and Databricks execution modes. · `omnigent/inner`
- [Transports](https://github.com/andokivilaid/omnigent/tree/main/omnigent/runner/transports): ws/tcp/uds · TCP, UDS, and WebSocket tunnel transports between server, runner, and hosts. · `omnigent/runner/transports`

### Persistence

Database-backed stores and schema used for session state.

- [Stores](https://github.com/andokivilaid/omnigent/tree/main/omnigent/stores): sqlalchemy stores · SQLAlchemy-backed stores for conversations, agents, files, comments, permissions, policies, and hosts. · `omnigent/stores`
- [DB schema](https://github.com/andokivilaid/omnigent/tree/main/omnigent/db): alembic models · Database models, migrations, and schema evolution for persistent session state. · `omnigent/db`

### External systems

Model providers, sandboxes, auth, and tool servers outside the repo.

- [LLM adapters](https://github.com/andokivilaid/omnigent/tree/main/omnigent/llms/adapters): provider adapters · Provider-neutral adapters for Anthropic, OpenAI, Gemini, Bedrock, Vertex, Databricks, and gateways. · `omnigent/llms/adapters`
- [Sandboxes](https://github.com/andokivilaid/omnigent/tree/main/omnigent/sandbox): execution isolation · Local and managed sandbox backends such as seatbelt, bubblewrap, Modal, Daytona, Islo, CWSandbox, Kubernetes, and Docker. · `omnigent/sandbox`
- [External tools](https://github.com/andokivilaid/omnigent/blob/main/omnigent/tools/mcp.py): mcp/tool servers · Remote MCP servers, local tool callables, and other external tool execution targets. · `omnigent/tools/mcp.py`

## Connections

- CLI → Server: session API · CLI reads and mutates sessions through the server and subscribes to updates.
- Web UI → Server: live console · Web UI consumes HTTP APIs plus streaming updates from the server.
- Desktop app → Web UI: wraps UI · Electron hosts the same web application and adds native integration.
- Server → Auth: authenticate · Server delegates login and access control to auth and OIDC flows.
- Server → Sessions API: session routes · Session endpoints expose conversation state, forks, and live streams.
- Server → Collaboration: collaboration · Server exposes presence, comments, approvals, and sharing features.
- Server → Host API: host registry · Server manages hosts, tunnels, and remote runner attachment.
- Server → Stores: persist state · Server reads and writes canonical session resources through stores.
- Stores → DB schema: schema/migrations · Stores are backed by SQLAlchemy models and Alembic migrations.
- Runner → Runtime: execute workflow · Runner drives session workflows and emits structured session events.
- Runtime → Policies: enforce · Runtime gates actions through policy checks before execution.
- Runtime → Tools: invoke tools · Runtime dispatches tool calls and forwards structured results.
- Runtime → Harnesses: spawn agents · Runtime uses harness adapters to run different agent runtimes.
- Runner → Transports: connect · Runner exposes transports for local, remote, and tunneled communication.
- Runner → Server: stream events · Runner reports session progress and accepts control messages from server routes.
- Harnesses → LLM adapters: model calls · Harnesses use provider-neutral LLM clients via adapters.
- Harnesses → Sandboxes: isolate work · Harness execution is bound to sandboxed local or managed environments.
- Tools → External tools: MCP/local · Tools can dispatch to local callables or external MCP servers.
- Server → Runner: tunnel/attach · Server coordinates runner attachment and tunneled session connections.
- Web UI → Sessions API: stream updates · UI subscribes to live session and collaboration events for rendering.
- CLI → Runner: terminal session · CLI can attach to or launch execution-side sessions through the runner.