How bytedance/ui-tars-desktop works
This monorepo ships two related products: Agent TARS, a multimodal agent stack used through CLI and Web UI, and UI-TARS Desktop, a native GUI-agent app. Their user-facing workflows span task input, agent/tool-call handling, and browser or computer operations; Agent TARS also includes MCP and search capabilities. The graph preserves these documented branches and sampled core modules. Several runtime entry points and integrations were not sampled, so uncertain connections are omitted rather than inferred.
Main entry points: agent-tars.ts Agent Core · ComposableToolCallEngine.ts Engine Composer · prompt_t5.ts T5 Prompt Engine · main.ts Desktop App
Components 22
Agent TARS
- Agent CLI
multimodal/agent-tars/cli/src/index.ts - Agent Core
multimodal/agent-tars/core/src/agent-tars.ts - Agent Interface
multimodal/tarko/agent-interface/src/agent-event-stream.ts - Agent Web UI
multimodal/tarko/agent-ui/src - Tool Call Engine
multimodal/tarko/agent/src/tool-call-engine/PromptEngineeringToolCallEngine.ts
Omni Agent Engines
- Engine Composer
multimodal/omni-tars/core/src/ComposableToolCallEngine.ts - Engine Factory
multimodal/omni-tars/core/src/ComposableToolCallEngineFactory.ts - T5 Prompt Engine
multimodal/omni-tars/core/src/environments/prompt_t5.ts - Code Environment
multimodal/omni-tars/core/src/environments/code.ts
GUI Agent Stack
- GUI Tool Engine
multimodal/omni-tars/gui-agent/src/GUIAgentToolCallEngine.ts - GUI T5 Adapter
multimodal/omni-tars/gui-agent/src/GUIAgentT5Adapter.ts - GUI SDK Engine
multimodal/gui-agent/agent-sdk/src/ToolCallEngine.ts
Agent Tools
- Local Browser
multimodal/agent-tars/core/src/environments/local/browser/browser-manager.ts - Filesystem Tools
multimodal/agent-tars/core/src/environments/local/filesystem/filesystem-tools-manager.ts - Browser MCP
packages/agent-infra/mcp-servers/browser/src/server.ts - Browser Search
packages/agent-infra/search/browser-search/src/engines/get-search-engine.ts - Search Adapters
packages/agent-infra/search/browser-search/src/engines/google-engine.ts
UI-TARS Desktop
- Desktop App
apps/ui-tars/src/main/main.ts - Desktop Renderer
apps/ui-tars/src/renderer/src/App.tsx - Desktop Agent
apps/ui-tars/src/main/agent/operator.ts - Local And Remote Operators
apps/ui-tars/src/main/remote/operators.ts
Other components
- User
Connections 13
- User → Agent CLI: submits tasks
README.md - User → Agent Web UI: submits tasks
README.md - User → Desktop App: requests automation
README.md - Engine Composer → T5 Prompt Engine: imports engine
multimodal/omni-tars/core/src/ComposableToolCallEngine.ts - Engine Factory → Engine Composer: creates engine
multimodal/omni-tars/core/src/ComposableToolCallEngineFactory.ts - T5 Prompt Engine → Code Environment: imports prompts
multimodal/omni-tars/core/src/environments/prompt_t5.ts - GUI Tool Engine → GUI T5 Adapter: uses adapter
multimodal/omni-tars/gui-agent/src/GUIAgentToolCallEngine.ts - Agent Interface → Tool Call Engine: defines engine contract
- Browser Search → Search Adapters: selects adapter
packages/agent-infra/search/browser-search/src/engines/get-search-engine.ts - Agent Core → Local Browser: enables browser use
README.md - Agent Core → Filesystem Tools: enables tool use
README.md - Agent Core → Browser MCP: integrates MCP tools
README.md - Desktop App → Local And Remote Operators: offers operators
README.md