How qwenlm/qwen-agent works
1. Repository purpose
- This is the Qwen-Agent framework: a Python SDK for building LLM applications that combine chat, tool use, planning, memory, RAG, and multi-agent orchestration.
- The repo is not a single app; it is a platform with a core runtime, optional UI/server layers, examples, benchmarks, and documentation.
2. Core package: qwen_agent/
- This is the central architecture.
- It contains the generic agent abstraction (agent.py) plus a family of concrete agents in qwen_agent/agents/ such as Assistant, ReAct, function-calling agents, router/multi-agent variants, document-QA agents, writing agents, and simulator agents.
- These agents are the main orchestration layer that turns a user request into LLM calls, tool invocations, memory updates, and final answers.
3. LLM provider layer: qwen_agent/llm/
- The framework isolates model access behind provider adapters.
- It supports DashScope/Qwen APIs, OpenAI-compatible APIs, Azure, local/model-runtime backends like Transformers and OpenVINO, and multimodal variants for audio/VL/omni.
- This layer also owns function-calling support and prompt templates for tool-call parsing, which is important for diagramming the boundary between “agent logic” and “model transport/protocol.”
4. Tooling layer: qwen_agent/tools/
- Tools are first-class modules with a shared base interface and schema-driven registration.
- Built-in tools include web search, retrieval, document parsing, image generation/search/zoom, Python execution, code interpreter, MCP manager, storage, and external utilities like weather.
- The major boundary here is that tools are called by agents, but execute outside the LLM, often with their own I/O, storage, or sandbox constraints.
5. Retrieval and document pipeline
- RAG is implemented through dedicated retrieval and search subsystems: retrieval.py, search_tools/, document parsers, and doc vocabulary extraction.
- The flow is typically: ingest files/web content → parse/chunk/index → search or vector/hybrid retrieval → supply context back to the agent/LLM.
- This is a separate architecture slice from plain chat, and it appears in the doc-QA agents and the example applications.
6. Memory and state
- Conversation or agent memory is isolated in qwen_agent/memory/.
- Memory is part of the agent runtime, not the model layer: agents read/write state across turns to support long dialogues, virtual memory, and multi-step workflows.
- The architecture diagram should show memory as a shared state service used by several agents rather than a standalone app.
7. Multi-agent and workflow orchestration
- multi_agent_hub.py, router agents, group chat agents, and the writing/doc-QA specializations show that the framework supports composition of multiple roles.
- Common patterns include: routing a request to a specialized agent, sequentially decomposing tasks, or aggregating multiple subagent outputs.
- This is an orchestration boundary above LLM/tool primitives.
8. Sandboxed code execution
- The code interpreter and Python executor are separate execution environments, with Docker-based isolation for the code interpreter.
- This is a key security boundary: the agent can request code execution, but the actual runtime is external to the model and intentionally sandboxed.
- The repo also includes Docker image resources and kernel initialization helpers for this subsystem.
9. Serving and UI layers
- qwen_server/ provides server-side components for assistant/database/workstation-style services, with a small frontend bundle under the same tree.
- qwen_agent/gui/ provides a Gradio-based web UI for quickly wrapping any agent.
- These are presentation/hosting layers over the same core agent runtime; they do not replace the core orchestration model.
10. Browser and extension integration
- browser_qwen/ contains a browser extension-style assistant frontend, with content scripts, popup UI, and background script.
- This indicates the framework can be embedded into browser workflows, likely calling back into the agent runtime or server layer for assistance actions.
11. Benchmark suites
- benchmark/code_interpreter/ and benchmark/deepplanning/ are self-contained evaluation harnesses, not production features.
- Code Interpreter benchmark focuses on model/tool execution and metrics; DeepPlanning includes shopping and travel planning tasks with domain tools, evaluation pipelines, and result aggregation.
- For architecture diagrams, treat benchmarks as consumer workloads that exercise the same agent/tool/LLM stack under controlled datasets and metrics.
12. Examples and cookbook applications
- examples/ is a large catalog of end-to-end scenarios: RAG, audio, multimodal, function calling, long dialogue, math/tool-integrated reasoning, group chat, custom tools, and MCP.
- These examples are valuable because they reveal the intended integration patterns: agent + model config + tools + optional files/UI.
- They are not core subsystems, but they demonstrate how the framework is assembled in real deployments.
13. Documentation and release automation
- qwen-agent-docs/website is a separate Next.js documentation site with localized content and benchmark pages.
- .github/workflows/deploy-docs.yml shows docs publishing automation.
- This is infrastructure around the framework, not part of the runtime path.
14. Primary runtime data flow
- Typical flow: user message or external event → agent orchestration → LLM call → tool selection or function call → tool execution / retrieval / memory updates → optional follow-up LLM calls → response.
- For multimodal or document-driven cases, files/images/web content enter through tools or agent inputs first, then get parsed or indexed before being reasoned over.
- The key diagram boundaries are: frontend/UI, agent orchestration, LLM provider, tools/execution sandbox, retrieval/memory, and external services.
Main entry points: agents Agent family · examples Examples · agent.py Agent base · llm LLM adapters
Components 18
Core runtime
Agent orchestration, model adapters, tools, retrieval, and memory.
- Agent base · Shared agent abstraction and turn-by-turn orchestration contract.
qwen_agent/agent.py - Agent family · Specialized agents for chat, ReAct, routing, document QA, writing, memory, and simulation.
qwen_agent/agents - Multi-agent hub · Coordination layer for agent composition and shared dispatch patterns.
qwen_agent/multi_agent_hub.py - LLM adapters · Model backends and protocol adapters for DashScope, OpenAI-compatible, Azure, local runtimes, and multimodal variants.
qwen_agent/llm - Fn-call prompts · Prompt templates and helpers for function-call parsing and tool-call framing.
qwen_agent/llm/fncall_prompts - Tool base · Shared tool interface and schema registration for external capabilities.
qwen_agent/tools/base.py - Tool suite · Built-in tools for web search, retrieval, parsing, image ops, Python execution, code interpreter, MCP, storage, and utilities.
qwen_agent/tools - Retrieval pipeline · Document ingestion, parsing, indexing, and search helpers used by RAG and doc-QA flows.
qwen_agent/tools/retrieval.py - Search backends · Keyword, vector, hybrid, and front-page search implementations.
qwen_agent/tools/search_tools - Memory layer · Conversation and agent state across turns and workflows.
qwen_agent/memory/memory.py - Code execution · Python executor and code interpreter sandbox boundary, including Docker-based resources.
qwen_agent/tools/code_interpreter.py
Interfaces
Serving, UI, and browser entrypoints on top of the core runtime.
- Gradio UI · Lightweight web UI wrapper for interacting with agents.
qwen_agent/gui/web_ui.py - Qwen server · Assistant, database, and workstation server components with frontend assets.
qwen_server - Browser assistant · Content script, popup, and background extension frontend.
browser_qwen
Workloads
Examples and benchmarks that exercise the framework.
- Examples · End-to-end sample applications for chat, RAG, multimodal, tool use, routing, and multi-agent patterns.
examples - Code benchmark · Benchmark for code-interpreter style reasoning, execution, and metrics.
benchmark/code_interpreter - Planning benchmark · DeepPlanning shopping and travel evaluation pipelines with domain tools and scoring.
benchmark/deepplanning
Docs
Documentation site and publishing automation.
- Docs site · Next.js documentation site with guides, benchmarks, and localized content.
qwen-agent-docs/website
Connections 20
- Agent base → Agent family: implements · Concrete agents build on the shared agent abstraction.
- Agent family → LLM adapters: calls · Agents delegate reasoning and generation to model adapters.
- Agent family → Tool base: uses · Agents invoke tools through the common tool interface.
- Tool suite → Tool base: implements · Built-in tools conform to the shared contract.
- Agent family → Retrieval pipeline: uses · Doc-QA and RAG agents pull context through retrieval flows.
- Retrieval pipeline → Search backends: depends on · Retrieval combines search implementations and indexing strategies.
- Agent family → Memory layer: reads/writes · Agents persist dialogue and workflow state across turns.
- Agent family → Code execution: can invoke · Some agents request sandboxed code execution or notebook-like actions.
- Multi-agent hub → Agent family: coordinates · Multi-agent flows route, compose, and aggregate agent behavior.
- Gradio UI → Agent base: hosts · The UI wraps the core agent runtime for interactive use.
- Qwen server → Agent base: serves · Server endpoints expose the same agent runtime to external clients.
- Browser assistant → Qwen server: connects to · The extension frontend typically bridges to hosted assistance services.
- Examples → Agent family: demonstrates · Examples show how to assemble agents with models, tools, and memory.
- Examples → LLM adapters: configures · Example apps select model backends and prompt styles.
- Examples → Tool suite: uses · Examples exercise built-in and custom tools.
- Code benchmark → LLM adapters: evaluates · The benchmark measures model/tool interaction under controlled tasks.
- Code benchmark → Code execution: tests · Code-interpreter evaluation focuses on execution behavior and metrics.
- Planning benchmark → Agent family: evaluates · Planning tasks exercise agentic decomposition and tool use.
- Docs site → Examples: documents · Docs describe usage patterns and benchmark results.
- Docs site → Agent family: documents · Guides explain the core agent, LLM, tool, context, and RAG modules.