# 777genius/os-ai-computer-use architecture

> Architecture diagram of the GitHub repository 777genius/os-ai-computer-use, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-10-01). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/777genius/os-ai-computer-use
- Repository: https://github.com/777genius/os-ai-computer-use

## Overview

OS AI Computer Use is a local desktop-automation agent with CLI and backend entry points, a provider-neutral agent core, OpenAI/Anthropic model adapters, and cross-platform OS drivers. The main workflow accepts a task, repeatedly requests model responses, executes requested computer tools, and returns assistant output with usage information. The README also documents Flutter chat, sessions, uploads, settings, and cost tracking. Backend WebSocket internals and several core modules were not excerpted, so uncertain implementation links are omitted or marked unverified.

Main entry points: [`packages/backend/src/os_ai_backend/app.py`](https://github.com/777genius/os-ai-computer-use/blob/main/packages/backend/src/os_ai_backend/app.py) (HTTP and WebSocket API), [`packages/os/src/os_ai_os/platform/drivers.py`](https://github.com/777genius/os-ai-computer-use/blob/main/packages/os/src/os_ai_os/platform/drivers.py) (Platform drivers), [`packages/core/src/os_ai_core/orchestrator.py`](https://github.com/777genius/os-ai-computer-use/blob/main/packages/core/src/os_ai_core/orchestrator.py) (Agent orchestrator), [`packages/core/src/os_ai_core/adapters/legacy/orchestrator_runner.py`](https://github.com/777genius/os-ai-computer-use/blob/main/packages/core/src/os_ai_core/adapters/legacy/orchestrator_runner.py) (Legacy agent loop)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_entry["User entry points"]
  node_cli["CLI task runner<br/>[main.py]"]
end

subgraph group_backend["Backend services"]
  node_backend_app["HTTP and WebSocket API<br/>[app.py]"]
  node_ws_rpc["WebSocket RPC"]
  node_sessions[("Chat sessions<br/>[sessions.py]")]
  node_file_store[("File storage<br/>[files.py]")]
  node_settings_metrics["Settings and metrics<br/>[settings.py]"]
end

subgraph group_agent["Agent runtime"]
  node_orchestrator["Agent orchestrator<br/>[orchestrator.py]"]
  node_run_agent["Agent use case<br/>[run_agent.py]"]
  node_legacy_runner["Legacy agent loop"]
  node_llm_gateway["Legacy LLM gateway<br/>[legacy_gateway.py]"]
  node_tool_gateway["Composite tool gateway"]
  node_local_tools["Computer tool provider"]
  node_tool_registry["Tool registry<br/>[registry.py]"]
  node_mcp["MCP tool integration<br/>[tool_provider.py]"]
end

subgraph group_providers["Model providers"]
  node_llm_contract["LLM client contract<br/>[interfaces.py]"]
  node_anthropic["Anthropic adapter"]
  node_openai["OpenAI adapter<br/>[adapters_openai.py]"]
end

subgraph group_desktop["Desktop execution"]
  node_computer_tools["Computer actions<br/>[computer.py]"]
  node_os_ports["OS capability ports"]
  node_os_drivers["Platform drivers<br/>[drivers.py]"]
  node_linux_driver["Linux driver<br/>[drivers.py]"]
  node_macos_driver["macOS driver<br/>[drivers.py]"]
  node_windows_driver["Windows driver<br/>[drivers.py]"]
end

node_user(("User"))
node_desktop(("Local desktop"))
node_model_services(("Provider APIs"))

node_user -->|"submits task"| node_cli
node_cli -->|"runs agent"| node_orchestrator
node_user -->|"uses desktop UI"| node_backend_app
node_backend_app -->|"routes WebSocket"| node_ws_rpc
node_backend_app -->|"stores uploads"| node_file_store
node_backend_app -->|"updates settings"| node_settings_metrics
node_ws_rpc -.->|"dispatches sessions"| node_sessions
node_orchestrator -->|"executes use case"| node_run_agent
node_orchestrator -->|"runs legacy path"| node_legacy_runner
node_legacy_runner -->|"requests responses"| node_llm_gateway
node_legacy_runner -->|"executes calls"| node_tool_gateway
node_tool_gateway -.->|"dispatches tools"| node_local_tools
node_local_tools -.->|"uses registry"| node_tool_registry
node_llm_gateway -.->|"adapts client"| node_llm_contract
node_anthropic -->|"implements contract"| node_llm_contract
node_openai -->|"implements contract"| node_llm_contract
node_anthropic -->|"calls provider"| node_model_services
node_openai -->|"calls provider"| node_model_services
node_tool_gateway -.->|"may use tools"| node_mcp
node_local_tools -.->|"provides actions"| node_computer_tools
node_computer_tools -->|"controls desktop"| node_desktop
node_computer_tools -.->|"uses OS capabilities"| node_os_ports
node_os_drivers -->|"implements ports"| node_os_ports
node_linux_driver -->|"supplies driver"| node_os_drivers
node_macos_driver -.->|"supplies driver"| node_os_drivers
node_windows_driver -->|"supplies driver"| node_os_drivers

click node_cli "https://github.com/777genius/os-ai-computer-use/blob/main/packages/cli/src/os_ai_cli/main.py"
click node_backend_app "https://github.com/777genius/os-ai-computer-use/blob/main/packages/backend/src/os_ai_backend/app.py"
click node_ws_rpc "https://github.com/777genius/os-ai-computer-use/tree/main/packages/backend/src/os_ai_backend"
click node_sessions "https://github.com/777genius/os-ai-computer-use/blob/main/packages/backend/src/os_ai_backend/sessions.py"
click node_file_store "https://github.com/777genius/os-ai-computer-use/blob/main/packages/backend/src/os_ai_backend/files.py"
click node_settings_metrics "https://github.com/777genius/os-ai-computer-use/blob/main/packages/backend/src/os_ai_backend/settings.py"
click node_orchestrator "https://github.com/777genius/os-ai-computer-use/blob/main/packages/core/src/os_ai_core/orchestrator.py"
click node_run_agent "https://github.com/777genius/os-ai-computer-use/blob/main/packages/core/src/os_ai_core/application/use_cases/run_agent.py"
click node_legacy_runner "https://github.com/777genius/os-ai-computer-use/blob/main/packages/core/src/os_ai_core/adapters/legacy/orchestrator_runner.py"
click node_llm_gateway "https://github.com/777genius/os-ai-computer-use/blob/main/packages/core/src/os_ai_core/adapters/llm/legacy_gateway.py"
click node_tool_gateway "https://github.com/777genius/os-ai-computer-use/blob/main/packages/core/src/os_ai_core/adapters/tools/composite_tool_gateway.py"
click node_local_tools "https://github.com/777genius/os-ai-computer-use/blob/main/packages/core/src/os_ai_core/adapters/tools/local_computer_provider.py"
click node_tool_registry "https://github.com/777genius/os-ai-computer-use/blob/main/packages/core/src/os_ai_core/tools/registry.py"
click node_llm_contract "https://github.com/777genius/os-ai-computer-use/blob/main/packages/llm/src/os_ai_llm/interfaces.py"
click node_anthropic "https://github.com/777genius/os-ai-computer-use/blob/main/packages/llm_anthropic/src/os_ai_llm_anthropic/adapters_anthropic.py"
click node_openai "https://github.com/777genius/os-ai-computer-use/blob/main/packages/llm_openai/src/os_ai_llm_openai/adapters_openai.py"
click node_mcp "https://github.com/777genius/os-ai-computer-use/blob/main/packages/mcp/src/os_ai_mcp/client/tool_provider.py"
click node_computer_tools "https://github.com/777genius/os-ai-computer-use/blob/main/packages/core/src/os_ai_core/tools/computer.py"
click node_os_ports "https://github.com/777genius/os-ai-computer-use/tree/main/packages/os/src/os_ai_os/ports"
click node_os_drivers "https://github.com/777genius/os-ai-computer-use/blob/main/packages/os/src/os_ai_os/platform/drivers.py"
click node_linux_driver "https://github.com/777genius/os-ai-computer-use/blob/main/packages/os-linux/src/os_ai_os_linux/drivers.py"
click node_macos_driver "https://github.com/777genius/os-ai-computer-use/blob/main/packages/os-macos/src/os_ai_os_macos/drivers.py"
click node_windows_driver "https://github.com/777genius/os-ai-computer-use/blob/main/packages/os-windows/src/os_ai_os_windows/drivers.py"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_cli,node_user toneBlue
class node_backend_app,node_ws_rpc,node_sessions,node_file_store,node_settings_metrics toneAmber
class node_orchestrator,node_run_agent,node_legacy_runner,node_llm_gateway,node_tool_gateway,node_local_tools,node_tool_registry,node_mcp,node_model_services toneMint
class node_llm_contract,node_anthropic,node_openai toneRose
class node_computer_tools,node_os_ports,node_os_drivers,node_linux_driver,node_macos_driver,node_windows_driver,node_desktop toneIndigo
```

## Components

### User entry points

- [CLI task runner](https://github.com/777genius/os-ai-computer-use/blob/main/packages/cli/src/os_ai_cli/main.py): `packages/cli/src/os_ai_cli/main.py`

### Backend services

- [HTTP and WebSocket API](https://github.com/777genius/os-ai-computer-use/blob/main/packages/backend/src/os_ai_backend/app.py): `packages/backend/src/os_ai_backend/app.py`
- [WebSocket RPC](https://github.com/777genius/os-ai-computer-use/tree/main/packages/backend/src/os_ai_backend): `packages/backend/src/os_ai_backend`
- [Chat sessions](https://github.com/777genius/os-ai-computer-use/blob/main/packages/backend/src/os_ai_backend/sessions.py): `packages/backend/src/os_ai_backend/sessions.py`
- [File storage](https://github.com/777genius/os-ai-computer-use/blob/main/packages/backend/src/os_ai_backend/files.py): `packages/backend/src/os_ai_backend/files.py`
- [Settings and metrics](https://github.com/777genius/os-ai-computer-use/blob/main/packages/backend/src/os_ai_backend/settings.py): `packages/backend/src/os_ai_backend/settings.py`

### Agent runtime

- [Agent orchestrator](https://github.com/777genius/os-ai-computer-use/blob/main/packages/core/src/os_ai_core/orchestrator.py): `packages/core/src/os_ai_core/orchestrator.py`
- [Agent use case](https://github.com/777genius/os-ai-computer-use/blob/main/packages/core/src/os_ai_core/application/use_cases/run_agent.py): `packages/core/src/os_ai_core/application/use_cases/run_agent.py`
- [Legacy agent loop](https://github.com/777genius/os-ai-computer-use/blob/main/packages/core/src/os_ai_core/adapters/legacy/orchestrator_runner.py): `packages/core/src/os_ai_core/adapters/legacy/orchestrator_runner.py`
- [Legacy LLM gateway](https://github.com/777genius/os-ai-computer-use/blob/main/packages/core/src/os_ai_core/adapters/llm/legacy_gateway.py): `packages/core/src/os_ai_core/adapters/llm/legacy_gateway.py`
- [Composite tool gateway](https://github.com/777genius/os-ai-computer-use/blob/main/packages/core/src/os_ai_core/adapters/tools/composite_tool_gateway.py): `packages/core/src/os_ai_core/adapters/tools/composite_tool_gateway.py`
- [Computer tool provider](https://github.com/777genius/os-ai-computer-use/blob/main/packages/core/src/os_ai_core/adapters/tools/local_computer_provider.py): `packages/core/src/os_ai_core/adapters/tools/local_computer_provider.py`
- [Tool registry](https://github.com/777genius/os-ai-computer-use/blob/main/packages/core/src/os_ai_core/tools/registry.py): `packages/core/src/os_ai_core/tools/registry.py`
- [MCP tool integration](https://github.com/777genius/os-ai-computer-use/blob/main/packages/mcp/src/os_ai_mcp/client/tool_provider.py): `packages/mcp/src/os_ai_mcp/client/tool_provider.py`

### Model providers

- [LLM client contract](https://github.com/777genius/os-ai-computer-use/blob/main/packages/llm/src/os_ai_llm/interfaces.py): `packages/llm/src/os_ai_llm/interfaces.py`
- [Anthropic adapter](https://github.com/777genius/os-ai-computer-use/blob/main/packages/llm_anthropic/src/os_ai_llm_anthropic/adapters_anthropic.py): `packages/llm_anthropic/src/os_ai_llm_anthropic/adapters_anthropic.py`
- [OpenAI adapter](https://github.com/777genius/os-ai-computer-use/blob/main/packages/llm_openai/src/os_ai_llm_openai/adapters_openai.py): `packages/llm_openai/src/os_ai_llm_openai/adapters_openai.py`

### Desktop execution

- [Computer actions](https://github.com/777genius/os-ai-computer-use/blob/main/packages/core/src/os_ai_core/tools/computer.py): `packages/core/src/os_ai_core/tools/computer.py`
- [OS capability ports](https://github.com/777genius/os-ai-computer-use/tree/main/packages/os/src/os_ai_os/ports): `packages/os/src/os_ai_os/ports`
- [Platform drivers](https://github.com/777genius/os-ai-computer-use/blob/main/packages/os/src/os_ai_os/platform/drivers.py): `packages/os/src/os_ai_os/platform/drivers.py`
- [Linux driver](https://github.com/777genius/os-ai-computer-use/blob/main/packages/os-linux/src/os_ai_os_linux/drivers.py): `packages/os-linux/src/os_ai_os_linux/drivers.py`
- [macOS driver](https://github.com/777genius/os-ai-computer-use/blob/main/packages/os-macos/src/os_ai_os_macos/drivers.py): `packages/os-macos/src/os_ai_os_macos/drivers.py`
- [Windows driver](https://github.com/777genius/os-ai-computer-use/blob/main/packages/os-windows/src/os_ai_os_windows/drivers.py): `packages/os-windows/src/os_ai_os_windows/drivers.py`

### Other components

- **User**
- **Local desktop**
- **Provider APIs**

## Connections

- User → CLI task runner: submits task
- CLI task runner → Agent orchestrator: runs agent (evidence: `packages/cli/src/os_ai_cli/main.py`)
- User → HTTP and WebSocket API: uses desktop UI
- HTTP and WebSocket API → WebSocket RPC: routes WebSocket (evidence: `packages/backend/src/os_ai_backend/app.py`)
- HTTP and WebSocket API → File storage: stores uploads (evidence: `packages/backend/src/os_ai_backend/app.py`)
- HTTP and WebSocket API → Settings and metrics: updates settings (evidence: `packages/backend/src/os_ai_backend/app.py`)
- WebSocket RPC → Chat sessions: dispatches sessions
- Agent orchestrator → Agent use case: executes use case (evidence: `packages/core/src/os_ai_core/orchestrator.py`)
- Agent orchestrator → Legacy agent loop: runs legacy path (evidence: `packages/core/src/os_ai_core/orchestrator.py`)
- Legacy agent loop → Legacy LLM gateway: requests responses (evidence: `packages/core/src/os_ai_core/adapters/legacy/orchestrator_runner.py`)
- Legacy agent loop → Composite tool gateway: executes calls (evidence: `packages/core/src/os_ai_core/adapters/legacy/orchestrator_runner.py`)
- Composite tool gateway → Computer tool provider: dispatches tools
- Computer tool provider → Tool registry: uses registry
- Legacy LLM gateway → LLM client contract: adapts client (evidence: `packages/core/src/os_ai_core/adapters/llm/legacy_gateway.py`)
- Anthropic adapter → LLM client contract: implements contract (evidence: `packages/llm_anthropic/src/os_ai_llm_anthropic/adapters_anthropic.py`)
- OpenAI adapter → LLM client contract: implements contract (evidence: `packages/llm_openai/src/os_ai_llm_openai/adapters_openai.py`)
- Anthropic adapter → Provider APIs: calls provider
- OpenAI adapter → Provider APIs: calls provider
- Composite tool gateway → MCP tool integration: may use tools
- Computer tool provider → Computer actions: provides actions
- Computer actions → Local desktop: controls desktop
- Computer actions → OS capability ports: uses OS capabilities
- Platform drivers → OS capability ports: implements ports (evidence: `packages/os/src/os_ai_os/platform/drivers.py`)
- Linux driver → Platform drivers: supplies driver (evidence: `packages/os-linux/src/os_ai_os_linux/drivers.py`)
- macOS driver → Platform drivers: supplies driver
- Windows driver → Platform drivers: supplies driver (evidence: `packages/os-windows/src/os_ai_os_windows/drivers.py`)
