# bytedance/ui-tars-desktop architecture

> Architecture diagram of the GitHub repository bytedance/ui-tars-desktop, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-11-10). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/bytedance/ui-tars-desktop
- Repository: https://github.com/bytedance/ui-tars-desktop

## Overview

This monorepo ships two related products: Agent TARS, a multimodal agent stack used through CLI and Web UI, and UI-TARS Desktop, a native GUI-agent app. Their user-facing workflows span task input, agent/tool-call handling, and browser or computer operations; Agent TARS also includes MCP and search capabilities. The graph preserves these documented branches and sampled core modules. Several runtime entry points and integrations were not sampled, so uncertain connections are omitted rather than inferred.

Main entry points: [`multimodal/agent-tars/core/src/agent-tars.ts`](https://github.com/bytedance/ui-tars-desktop/blob/main/multimodal/agent-tars/core/src/agent-tars.ts) (Agent Core), [`multimodal/omni-tars/core/src/ComposableToolCallEngine.ts`](https://github.com/bytedance/ui-tars-desktop/blob/main/multimodal/omni-tars/core/src/ComposableToolCallEngine.ts) (Engine Composer), [`multimodal/omni-tars/core/src/environments/prompt_t5.ts`](https://github.com/bytedance/ui-tars-desktop/blob/main/multimodal/omni-tars/core/src/environments/prompt_t5.ts) (T5 Prompt Engine), [`apps/ui-tars/src/main/main.ts`](https://github.com/bytedance/ui-tars-desktop/blob/main/apps/ui-tars/src/main/main.ts) (Desktop App)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_agent["Agent TARS"]
  node_agent_cli["Agent CLI<br/>[index.ts]"]
  node_agent_core["Agent Core<br/>[agent-tars.ts]"]
  node_agent_interface["Agent Interface"]
  node_web_ui["Agent Web UI"]
  node_tool_engine["Tool Call Engine"]
end

subgraph group_omni["Omni Agent Engines"]
  node_omni_composer["Engine Composer"]
  node_omni_factory["Engine Factory"]
  node_prompt_t5["T5 Prompt Engine<br/>[prompt_t5.ts]"]
  node_code_environment["Code Environment<br/>[code.ts]"]
end

subgraph group_gui["GUI Agent Stack"]
  node_gui_engine["GUI Tool Engine"]
  node_gui_adapter["GUI T5 Adapter"]
  node_gui_sdk_engine["GUI SDK Engine<br/>[ToolCallEngine.ts]"]
end

subgraph group_infra["Agent Tools"]
  node_local_browser["Local Browser<br/>[browser-manager.ts]"]
  node_filesystem["Filesystem Tools"]
  node_mcp_browser["Browser MCP<br/>[server.ts]"]
  node_browser_search["Browser Search"]
  node_search_adapters["Search Adapters<br/>[google-engine.ts]"]
end

subgraph group_desktop["UI-TARS Desktop"]
  node_ui_desktop["Desktop App<br/>[main.ts]"]
  node_ui_renderer["Desktop Renderer<br/>[App.tsx]"]
  node_ui_agent["Desktop Agent<br/>[operator.ts]"]
  node_local_remote_ops["Local And Remote Operators<br/>[operators.ts]"]
end

node_user(("User"))

node_user -->|"submits tasks"| node_agent_cli
node_user -->|"submits tasks"| node_web_ui
node_user -->|"requests automation"| node_ui_desktop
node_omni_composer -->|"imports engine"| node_prompt_t5
node_omni_factory -->|"creates engine"| node_omni_composer
node_prompt_t5 -->|"imports prompts"| node_code_environment
node_gui_engine -->|"uses adapter"| node_gui_adapter
node_agent_interface -->|"defines engine contract"| node_tool_engine
node_browser_search -->|"selects adapter"| node_search_adapters
node_agent_core -.->|"enables browser use"| node_local_browser
node_agent_core -.->|"enables tool use"| node_filesystem
node_agent_core -.->|"integrates MCP tools"| node_mcp_browser
node_ui_desktop -->|"offers operators"| node_local_remote_ops

click node_agent_cli "https://github.com/bytedance/ui-tars-desktop/blob/main/multimodal/agent-tars/cli/src/index.ts"
click node_agent_core "https://github.com/bytedance/ui-tars-desktop/blob/main/multimodal/agent-tars/core/src/agent-tars.ts"
click node_agent_interface "https://github.com/bytedance/ui-tars-desktop/blob/main/multimodal/tarko/agent-interface/src/agent-event-stream.ts"
click node_web_ui "https://github.com/bytedance/ui-tars-desktop/tree/main/multimodal/tarko/agent-ui/src"
click node_tool_engine "https://github.com/bytedance/ui-tars-desktop/blob/main/multimodal/tarko/agent/src/tool-call-engine/PromptEngineeringToolCallEngine.ts"
click node_omni_composer "https://github.com/bytedance/ui-tars-desktop/blob/main/multimodal/omni-tars/core/src/ComposableToolCallEngine.ts"
click node_omni_factory "https://github.com/bytedance/ui-tars-desktop/blob/main/multimodal/omni-tars/core/src/ComposableToolCallEngineFactory.ts"
click node_prompt_t5 "https://github.com/bytedance/ui-tars-desktop/blob/main/multimodal/omni-tars/core/src/environments/prompt_t5.ts"
click node_code_environment "https://github.com/bytedance/ui-tars-desktop/blob/main/multimodal/omni-tars/core/src/environments/code.ts"
click node_gui_engine "https://github.com/bytedance/ui-tars-desktop/blob/main/multimodal/omni-tars/gui-agent/src/GUIAgentToolCallEngine.ts"
click node_gui_adapter "https://github.com/bytedance/ui-tars-desktop/blob/main/multimodal/omni-tars/gui-agent/src/GUIAgentT5Adapter.ts"
click node_gui_sdk_engine "https://github.com/bytedance/ui-tars-desktop/blob/main/multimodal/gui-agent/agent-sdk/src/ToolCallEngine.ts"
click node_local_browser "https://github.com/bytedance/ui-tars-desktop/blob/main/multimodal/agent-tars/core/src/environments/local/browser/browser-manager.ts"
click node_filesystem "https://github.com/bytedance/ui-tars-desktop/blob/main/multimodal/agent-tars/core/src/environments/local/filesystem/filesystem-tools-manager.ts"
click node_mcp_browser "https://github.com/bytedance/ui-tars-desktop/blob/main/packages/agent-infra/mcp-servers/browser/src/server.ts"
click node_browser_search "https://github.com/bytedance/ui-tars-desktop/blob/main/packages/agent-infra/search/browser-search/src/engines/get-search-engine.ts"
click node_search_adapters "https://github.com/bytedance/ui-tars-desktop/blob/main/packages/agent-infra/search/browser-search/src/engines/google-engine.ts"
click node_ui_desktop "https://github.com/bytedance/ui-tars-desktop/blob/main/apps/ui-tars/src/main/main.ts"
click node_ui_renderer "https://github.com/bytedance/ui-tars-desktop/blob/main/apps/ui-tars/src/renderer/src/App.tsx"
click node_ui_agent "https://github.com/bytedance/ui-tars-desktop/blob/main/apps/ui-tars/src/main/agent/operator.ts"
click node_local_remote_ops "https://github.com/bytedance/ui-tars-desktop/blob/main/apps/ui-tars/src/main/remote/operators.ts"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_agent_cli,node_agent_core,node_agent_interface,node_web_ui,node_tool_engine,node_user toneBlue
class node_omni_composer,node_omni_factory,node_prompt_t5,node_code_environment toneAmber
class node_gui_engine,node_gui_adapter,node_gui_sdk_engine toneMint
class node_local_browser,node_filesystem,node_mcp_browser,node_browser_search,node_search_adapters toneRose
class node_ui_desktop,node_ui_renderer,node_ui_agent,node_local_remote_ops toneIndigo
```

## Components

### Agent TARS

- [Agent CLI](https://github.com/bytedance/ui-tars-desktop/blob/main/multimodal/agent-tars/cli/src/index.ts): `multimodal/agent-tars/cli/src/index.ts`
- [Agent Core](https://github.com/bytedance/ui-tars-desktop/blob/main/multimodal/agent-tars/core/src/agent-tars.ts): `multimodal/agent-tars/core/src/agent-tars.ts`
- [Agent Interface](https://github.com/bytedance/ui-tars-desktop/blob/main/multimodal/tarko/agent-interface/src/agent-event-stream.ts): `multimodal/tarko/agent-interface/src/agent-event-stream.ts`
- [Agent Web UI](https://github.com/bytedance/ui-tars-desktop/tree/main/multimodal/tarko/agent-ui/src): `multimodal/tarko/agent-ui/src`
- [Tool Call Engine](https://github.com/bytedance/ui-tars-desktop/blob/main/multimodal/tarko/agent/src/tool-call-engine/PromptEngineeringToolCallEngine.ts): `multimodal/tarko/agent/src/tool-call-engine/PromptEngineeringToolCallEngine.ts`

### Omni Agent Engines

- [Engine Composer](https://github.com/bytedance/ui-tars-desktop/blob/main/multimodal/omni-tars/core/src/ComposableToolCallEngine.ts): `multimodal/omni-tars/core/src/ComposableToolCallEngine.ts`
- [Engine Factory](https://github.com/bytedance/ui-tars-desktop/blob/main/multimodal/omni-tars/core/src/ComposableToolCallEngineFactory.ts): `multimodal/omni-tars/core/src/ComposableToolCallEngineFactory.ts`
- [T5 Prompt Engine](https://github.com/bytedance/ui-tars-desktop/blob/main/multimodal/omni-tars/core/src/environments/prompt_t5.ts): `multimodal/omni-tars/core/src/environments/prompt_t5.ts`
- [Code Environment](https://github.com/bytedance/ui-tars-desktop/blob/main/multimodal/omni-tars/core/src/environments/code.ts): `multimodal/omni-tars/core/src/environments/code.ts`

### GUI Agent Stack

- [GUI Tool Engine](https://github.com/bytedance/ui-tars-desktop/blob/main/multimodal/omni-tars/gui-agent/src/GUIAgentToolCallEngine.ts): `multimodal/omni-tars/gui-agent/src/GUIAgentToolCallEngine.ts`
- [GUI T5 Adapter](https://github.com/bytedance/ui-tars-desktop/blob/main/multimodal/omni-tars/gui-agent/src/GUIAgentT5Adapter.ts): `multimodal/omni-tars/gui-agent/src/GUIAgentT5Adapter.ts`
- [GUI SDK Engine](https://github.com/bytedance/ui-tars-desktop/blob/main/multimodal/gui-agent/agent-sdk/src/ToolCallEngine.ts): `multimodal/gui-agent/agent-sdk/src/ToolCallEngine.ts`

### Agent Tools

- [Local Browser](https://github.com/bytedance/ui-tars-desktop/blob/main/multimodal/agent-tars/core/src/environments/local/browser/browser-manager.ts): `multimodal/agent-tars/core/src/environments/local/browser/browser-manager.ts`
- [Filesystem Tools](https://github.com/bytedance/ui-tars-desktop/blob/main/multimodal/agent-tars/core/src/environments/local/filesystem/filesystem-tools-manager.ts): `multimodal/agent-tars/core/src/environments/local/filesystem/filesystem-tools-manager.ts`
- [Browser MCP](https://github.com/bytedance/ui-tars-desktop/blob/main/packages/agent-infra/mcp-servers/browser/src/server.ts): `packages/agent-infra/mcp-servers/browser/src/server.ts`
- [Browser Search](https://github.com/bytedance/ui-tars-desktop/blob/main/packages/agent-infra/search/browser-search/src/engines/get-search-engine.ts): `packages/agent-infra/search/browser-search/src/engines/get-search-engine.ts`
- [Search Adapters](https://github.com/bytedance/ui-tars-desktop/blob/main/packages/agent-infra/search/browser-search/src/engines/google-engine.ts): `packages/agent-infra/search/browser-search/src/engines/google-engine.ts`

### UI-TARS Desktop

- [Desktop App](https://github.com/bytedance/ui-tars-desktop/blob/main/apps/ui-tars/src/main/main.ts): `apps/ui-tars/src/main/main.ts`
- [Desktop Renderer](https://github.com/bytedance/ui-tars-desktop/blob/main/apps/ui-tars/src/renderer/src/App.tsx): `apps/ui-tars/src/renderer/src/App.tsx`
- [Desktop Agent](https://github.com/bytedance/ui-tars-desktop/blob/main/apps/ui-tars/src/main/agent/operator.ts): `apps/ui-tars/src/main/agent/operator.ts`
- [Local And Remote Operators](https://github.com/bytedance/ui-tars-desktop/blob/main/apps/ui-tars/src/main/remote/operators.ts): `apps/ui-tars/src/main/remote/operators.ts`

### Other components

- **User**

## Connections

- User → Agent CLI: submits tasks (evidence: `README.md`)
- User → Agent Web UI: submits tasks (evidence: `README.md`)
- User → Desktop App: requests automation (evidence: `README.md`)
- Engine Composer → T5 Prompt Engine: imports engine (evidence: `multimodal/omni-tars/core/src/ComposableToolCallEngine.ts`)
- Engine Factory → Engine Composer: creates engine (evidence: `multimodal/omni-tars/core/src/ComposableToolCallEngineFactory.ts`)
- T5 Prompt Engine → Code Environment: imports prompts (evidence: `multimodal/omni-tars/core/src/environments/prompt_t5.ts`)
- GUI Tool Engine → GUI T5 Adapter: uses adapter (evidence: `multimodal/omni-tars/gui-agent/src/GUIAgentToolCallEngine.ts`)
- Agent Interface → Tool Call Engine: defines engine contract
- Browser Search → Search Adapters: selects adapter (evidence: `packages/agent-infra/search/browser-search/src/engines/get-search-engine.ts`)
- Agent Core → Local Browser: enables browser use (evidence: `README.md`)
- Agent Core → Filesystem Tools: enables tool use (evidence: `README.md`)
- Agent Core → Browser MCP: integrates MCP tools (evidence: `README.md`)
- Desktop App → Local And Remote Operators: offers operators (evidence: `README.md`)
