# qwenlm/qwen-mm-plugins architecture

> Architecture diagram of the GitHub repository qwenlm/qwen-mm-plugins, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-08-09). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/qwenlm/qwen-mm-plugins
- Repository: https://github.com/qwenlm/qwen-mm-plugins

## Overview

#### Purpose and Delivery Model
Qwen-MM-Plugins is a Python-based marketplace of independently installable multimodal agent capabilities. A capability combines agent instructions (“skill”) with an optional stdio MCP server exposing tools. It targets multiple agent harnesses rather than providing a standalone web application. Main distribution/install entry points are `install.sh`, `.claude-plugin/marketplace.json`, and per-capability plugin manifests such as `src/capabilities/core/.claude-plugin/plugin.json`.

#### Runtime Entry Points and MCP Boundary
Python MCP servers are launched on demand, generally through each package’s module entry point, e.g. `src/capabilities/core/qwen_mm_plugins_core/__main__.py` and `src/capabilities/video-memory/qwen_mm_plugins_video_memory/__main__.py`. `src/mcp_framework.py` is the likely shared server/tool-registration abstraction; stdio response streaming is implemented for core at `src/capabilities/core/qwen_mm_plugins_core/stdio_streaming.py`. Skills define model-facing workflow and tool-use guidance independently of the executable server, e.g. `src/capabilities/core/skill/SKILL.md`.

#### Shared Platform Layer
Cross-capability infrastructure centralizes configuration, API access, execution, caching, and media utilities:
- Environment/config resolution: `src/shared/env.py`
- DashScope and OpenAI-compatible API clients: `src/shared/api_dashscope.py`, `src/shared/api_openai.py`
- Process, retry, cache, and content/image/video utilities: `src/shared/syscmd.py`, `src/shared/retry.py`, `src/shared/video.py`

Configuration prefers process environment and supports a shared user config file. External API boundaries include DashScope for perception/generation and Serper for web/search functions; native rendering and parsing additionally depend on local system tools.

#### Core Multimodal Processing
The `core` capability is the base perception and artifact-rendering layer. Readers ingest images and videos; API adapters provide OCR, ASR, visual grounding, segmentation, vision chat, and web retrieval. Producers create derivative views such as crops or bounding-box overlays. Renderers turn diverse source artifacts—PDF, Office, notebooks, HTML, LaTeX, SVG, geo data, and 3D models—into inspectable output.
Paths: `src/capabilities/core/qwen_mm_plugins_core/readers/video.py`, `src/capabilities/core/qwen_mm_plugins_core/apis/vision_chat.py`, `src/capabilities/core/qwen_mm_plugins_core/renderers/`.

#### Long-Video Memory Pipeline
Video memory creates and queries a hierarchical event graph for long-video QA. The build workflow coordinates extraction/analysis, embeddings, graph assembly, and merging; the runtime server loads this representation and exposes temporal, textual, summary, and subgraph queries. This is a persistent-data boundary: build artifacts are separate from request-time querying.
Paths: `src/capabilities/video-memory/skill/script/build_memory/pipeline_worker.py`, `src/capabilities/video-memory/skill/script/build_memory/build_graph.py`, `src/capabilities/video-memory/qwen_mm_plugins_video_memory/query_memory.py`.

#### Creative and Education Workflows
Video editing combines model-backed image/video/speech generation tools with skill-driven editing plans, FFmpeg-oriented execution guidance, media assets, and review gates. Educational content is skill-only: it guides image/problem analysis through scripting, narration, storyboard, component composition, rendering, and validation for Chinese explainer videos/pages.
Paths: `src/capabilities/video-edit/qwen_mm_plugins_video_edit/tools/wan_t2v.py`, `src/capabilities/video-edit/skill/engines/ffmpeg-direct.md`, `src/capabilities/edu-agent/skill/scripts/_render.py`.

#### Stateful 3D/CAD Thin Clients
Blender and FreeCAD capabilities are clients for already-running desktop applications, not embedded geometry engines. Their loaders/launchers establish the application boundary; MCP tools execute code and manipulate or inspect the current scene/document. Blender additionally integrates external asset catalogs and 3D generation services; FreeCAD supports parametric operations and FEM analysis.
Paths: `src/capabilities/blender/qwen_mm_plugins_blender/_launch.py`, `src/capabilities/blender/qwen_mm_plugins_blender/tools/execute_blender_code.py`, `src/capabilities/freecad/qwen_mm_plugins_freecad/tools/execute_code_async.py`.

#### Infrastructure Dependencies
The intended runtime is Python packaged via `pyproject.toml` and launched with `uvx`. Local dependencies materially affect capabilities: FFmpeg for audio/video; LibreOffice, Blender, TeX Live, and Chromium for selected render/visualization paths; running Blender/FreeCAD instances for their respective thin clients. Third-party API credentials gate DashScope-backed capabilities and Serper-backed web retrieval.

Main entry points: [`src/capabilities/core/qwen_mm_plugins_core/__main__.py`](https://github.com/QwenLM/Qwen-MM-Plugins/blob/main/src/capabilities/core/qwen_mm_plugins_core/__main__.py) (Core MCP server), [`src/capabilities/core/.claude-plugin/plugin.json`](https://github.com/QwenLM/Qwen-MM-Plugins/blob/main/src/capabilities/core/.claude-plugin/plugin.json) (Core plugin manifest), [`src/capabilities/core/qwen_mm_plugins_core/apis/vision_chat.py`](https://github.com/QwenLM/Qwen-MM-Plugins/blob/main/src/capabilities/core/qwen_mm_plugins_core/apis/vision_chat.py) (Vision API adapters), [`src/shared/env.py`](https://github.com/QwenLM/Qwen-MM-Plugins/blob/main/src/shared/env.py) (Shared runtime services)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_distribution["Distribution"]
  node_installer["Installer<br/>shell entry point<br/>[install.sh]"]
  node_marketplace["Marketplace manifest<br/>plugin catalog<br/>[marketplace.json]"]
  node_core_manifest["Core plugin manifest<br/>capability manifest<br/>[plugin.json]"]
end

subgraph group_platform["Shared platform"]
  node_mcp_framework{{"MCP framework<br/>shared server abstraction<br/>[mcp_framework.py]"}}
  node_shared_runtime["Shared runtime services<br/>[env.py]"]
end

subgraph group_core["Core capability"]
  node_core_skill["Core skill<br/>agent instructions<br/>[SKILL.md]"]
  node_core_server{{"Core MCP server<br/>Python module entry point<br/>[__main__.py]"}}
  node_core_readers["Media readers<br/>ingestion layer<br/>[video.py]"]
  node_core_vision_api["Vision API adapters<br/>perception tools<br/>[vision_chat.py]"]
  node_core_renderers["Artifact renderers<br/>rendering layer"]
end

subgraph group_video_memory["Video memory"]
  node_memory_builder["Memory build worker<br/>offline pipeline coordinator<br/>[pipeline_worker.py]"]
  node_memory_graph[("Event graph builder<br/>persistent graph assembly<br/>[build_graph.py]")]
  node_memory_query{{"Memory query service<br/>runtime query layer<br/>[query_memory.py]"}}
end

subgraph group_workflows["Creative &amp; desktop capabilities"]
  node_video_edit["Video editing tools<br/>generative media workflow<br/>[wan_t2v.py]"]
  node_ffmpeg_engine["FFmpeg editing engine<br/>execution guidance<br/>[ffmpeg-direct.md]"]
  node_edu_render["Education renderer<br/>skill-only workflow script<br/>[_render.py]"]
  node_blender_client{{"Blender thin client<br/>desktop MCP client<br/>[_launch.py]"}}
  node_blender_execute["Blender code tools<br/>scene execution tools"]
  node_freecad_client{{"FreeCAD thin client<br/>desktop MCP client"}}
end

subgraph group_external["External dependencies"]
  node_model_search_apis(("DashScope &amp; Serper<br/>credentialed external APIs"))
  node_local_tools(("Local tools &amp; desktop apps<br/>system runtime dependencies"))
end

node_installer -->|"installs from"| node_marketplace
node_marketplace -->|"lists capability"| node_core_manifest
node_core_manifest -->|"packages"| node_core_skill
node_core_manifest -->|"launches"| node_core_server
node_core_server -->|"uses"| node_mcp_framework
node_core_server -->|"exposes"| node_core_readers
node_core_server -->|"exposes"| node_core_vision_api
node_core_server -->|"exposes"| node_core_renderers
node_core_readers -->|"uses"| node_shared_runtime
node_core_vision_api -->|"uses"| node_shared_runtime
node_core_vision_api -->|"calls"| node_model_search_apis
node_core_renderers -->|"invokes"| node_local_tools
node_memory_builder -->|"builds"| node_memory_graph
node_memory_graph -->|"persistent artifacts"| node_memory_query
node_memory_builder -->|"uses APIs and media utilities"| node_shared_runtime
node_memory_query -->|"serves via MCP"| node_mcp_framework
node_video_edit -->|"generates media through"| node_model_search_apis
node_video_edit -->|"uses execution plan"| node_ffmpeg_engine
node_ffmpeg_engine -->|"requires FFmpeg"| node_local_tools
node_edu_render -->|"renders with local tools"| node_local_tools
node_blender_client -->|"provides session to"| node_blender_execute
node_blender_execute -->|"targets running Blender"| node_local_tools
node_freecad_client -->|"targets running FreeCAD"| node_local_tools

click node_installer "https://github.com/QwenLM/Qwen-MM-Plugins/blob/main/install.sh"
click node_marketplace "https://github.com/QwenLM/Qwen-MM-Plugins/blob/main/.claude-plugin/marketplace.json"
click node_core_manifest "https://github.com/QwenLM/Qwen-MM-Plugins/blob/main/src/capabilities/core/.claude-plugin/plugin.json"
click node_mcp_framework "https://github.com/QwenLM/Qwen-MM-Plugins/blob/main/src/mcp_framework.py"
click node_shared_runtime "https://github.com/QwenLM/Qwen-MM-Plugins/blob/main/src/shared/env.py"
click node_core_skill "https://github.com/QwenLM/Qwen-MM-Plugins/blob/main/src/capabilities/core/skill/SKILL.md"
click node_core_server "https://github.com/QwenLM/Qwen-MM-Plugins/blob/main/src/capabilities/core/qwen_mm_plugins_core/__main__.py"
click node_core_readers "https://github.com/QwenLM/Qwen-MM-Plugins/blob/main/src/capabilities/core/qwen_mm_plugins_core/readers/video.py"
click node_core_vision_api "https://github.com/QwenLM/Qwen-MM-Plugins/blob/main/src/capabilities/core/qwen_mm_plugins_core/apis/vision_chat.py"
click node_memory_builder "https://github.com/QwenLM/Qwen-MM-Plugins/blob/main/src/capabilities/video-memory/skill/script/build_memory/pipeline_worker.py"
click node_memory_graph "https://github.com/QwenLM/Qwen-MM-Plugins/blob/main/src/capabilities/video-memory/skill/script/build_memory/build_graph.py"
click node_memory_query "https://github.com/QwenLM/Qwen-MM-Plugins/blob/main/src/capabilities/video-memory/qwen_mm_plugins_video_memory/query_memory.py"
click node_video_edit "https://github.com/QwenLM/Qwen-MM-Plugins/blob/main/src/capabilities/video-edit/qwen_mm_plugins_video_edit/tools/wan_t2v.py"
click node_ffmpeg_engine "https://github.com/QwenLM/Qwen-MM-Plugins/blob/main/src/capabilities/video-edit/skill/engines/ffmpeg-direct.md"
click node_edu_render "https://github.com/QwenLM/Qwen-MM-Plugins/blob/main/src/capabilities/edu-agent/skill/scripts/_render.py"
click node_blender_client "https://github.com/QwenLM/Qwen-MM-Plugins/blob/main/src/capabilities/blender/qwen_mm_plugins_blender/_launch.py"
click node_blender_execute "https://github.com/QwenLM/Qwen-MM-Plugins/blob/main/src/capabilities/blender/qwen_mm_plugins_blender/tools/execute_blender_code.py"
click node_freecad_client "https://github.com/QwenLM/Qwen-MM-Plugins/blob/main/src/capabilities/freecad/qwen_mm_plugins_freecad/tools/execute_code_async.py"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_installer,node_marketplace,node_core_manifest toneBlue
class node_mcp_framework,node_shared_runtime toneAmber
class node_core_skill,node_core_server,node_core_readers,node_core_vision_api,node_core_renderers toneMint
class node_memory_builder,node_memory_graph,node_memory_query toneRose
class node_video_edit,node_ffmpeg_engine,node_edu_render,node_blender_client,node_blender_execute,node_freecad_client toneIndigo
class node_model_search_apis,node_local_tools toneTeal
```

## Components

### Distribution

- [Installer](https://github.com/QwenLM/Qwen-MM-Plugins/blob/main/install.sh): shell entry point · Installs marketplace capabilities for agent harnesses. · `install.sh`
- [Marketplace manifest](https://github.com/QwenLM/Qwen-MM-Plugins/blob/main/.claude-plugin/marketplace.json): plugin catalog · `.claude-plugin/marketplace.json`
- [Core plugin manifest](https://github.com/QwenLM/Qwen-MM-Plugins/blob/main/src/capabilities/core/.claude-plugin/plugin.json): capability manifest · `src/capabilities/core/.claude-plugin/plugin.json`

### Shared platform

- [MCP framework](https://github.com/QwenLM/Qwen-MM-Plugins/blob/main/src/mcp_framework.py): shared server abstraction · Registers tools and provides the stdio MCP boundary. · `src/mcp_framework.py`
- [Shared runtime services](https://github.com/QwenLM/Qwen-MM-Plugins/blob/main/src/shared/env.py): config, API & media utilities · Resolves environment, calls model APIs, and supplies process, retry, cache, and media helpers. · `src/shared/env.py`

### Core capability

- [Core skill](https://github.com/QwenLM/Qwen-MM-Plugins/blob/main/src/capabilities/core/skill/SKILL.md): agent instructions · Guides model workflow independently of the MCP server. · `src/capabilities/core/skill/SKILL.md`
- [Core MCP server](https://github.com/QwenLM/Qwen-MM-Plugins/blob/main/src/capabilities/core/qwen_mm_plugins_core/__main__.py): Python module entry point · On-demand stdio server for base multimodal tools. · `src/capabilities/core/qwen_mm_plugins_core/__main__.py`
- [Media readers](https://github.com/QwenLM/Qwen-MM-Plugins/blob/main/src/capabilities/core/qwen_mm_plugins_core/readers/video.py): ingestion layer · Ingests images and video for core processing. · `src/capabilities/core/qwen_mm_plugins_core/readers/video.py`
- [Vision API adapters](https://github.com/QwenLM/Qwen-MM-Plugins/blob/main/src/capabilities/core/qwen_mm_plugins_core/apis/vision_chat.py): perception tools · Provides vision chat and related model-backed perception. · `src/capabilities/core/qwen_mm_plugins_core/apis/vision_chat.py`
- **Artifact renderers**: rendering layer · Converts documents, web, geo, and 3D artifacts into inspectable output.

### Video memory

- [Memory build worker](https://github.com/QwenLM/Qwen-MM-Plugins/blob/main/src/capabilities/video-memory/skill/script/build_memory/pipeline_worker.py): offline pipeline coordinator · Coordinates extraction, analysis, embeddings, graph assembly, and merge. · `src/capabilities/video-memory/skill/script/build_memory/pipeline_worker.py`
- [Event graph builder](https://github.com/QwenLM/Qwen-MM-Plugins/blob/main/src/capabilities/video-memory/skill/script/build_memory/build_graph.py): persistent graph assembly · Creates the hierarchical long-video memory representation. · `src/capabilities/video-memory/skill/script/build_memory/build_graph.py`
- [Memory query service](https://github.com/QwenLM/Qwen-MM-Plugins/blob/main/src/capabilities/video-memory/qwen_mm_plugins_video_memory/query_memory.py): runtime query layer · Loads persistent memory for temporal, textual, summary, and subgraph queries. · `src/capabilities/video-memory/qwen_mm_plugins_video_memory/query_memory.py`

### Creative & desktop capabilities

- [Video editing tools](https://github.com/QwenLM/Qwen-MM-Plugins/blob/main/src/capabilities/video-edit/qwen_mm_plugins_video_edit/tools/wan_t2v.py): generative media workflow · Model-backed video generation supports skill-driven editing and review. · `src/capabilities/video-edit/qwen_mm_plugins_video_edit/tools/wan_t2v.py`
- [FFmpeg editing engine](https://github.com/QwenLM/Qwen-MM-Plugins/blob/main/src/capabilities/video-edit/skill/engines/ffmpeg-direct.md): execution guidance · `src/capabilities/video-edit/skill/engines/ffmpeg-direct.md`
- [Education renderer](https://github.com/QwenLM/Qwen-MM-Plugins/blob/main/src/capabilities/edu-agent/skill/scripts/_render.py): skill-only workflow script · Renders educational explainer video or page artifacts. · `src/capabilities/edu-agent/skill/scripts/_render.py`
- [Blender thin client](https://github.com/QwenLM/Qwen-MM-Plugins/blob/main/src/capabilities/blender/qwen_mm_plugins_blender/_launch.py): desktop MCP client · Launches or connects to a running Blender session. · `src/capabilities/blender/qwen_mm_plugins_blender/_launch.py`
- [Blender code tools](https://github.com/QwenLM/Qwen-MM-Plugins/blob/main/src/capabilities/blender/qwen_mm_plugins_blender/tools/execute_blender_code.py): scene execution tools · Manipulates and inspects the current Blender scene. · `src/capabilities/blender/qwen_mm_plugins_blender/tools/execute_blender_code.py`
- [FreeCAD thin client](https://github.com/QwenLM/Qwen-MM-Plugins/blob/main/src/capabilities/freecad/qwen_mm_plugins_freecad/tools/execute_code_async.py): desktop MCP client · Executes asynchronous code against the current FreeCAD document. · `src/capabilities/freecad/qwen_mm_plugins_freecad/tools/execute_code_async.py`

### External dependencies

- **DashScope & Serper**: credentialed external APIs · Perception, generation, and web retrieval boundary.
- **Local tools & desktop apps**: system runtime dependencies · FFmpeg, renderers, and running Blender or FreeCAD instances.

## Connections

- Installer → Marketplace manifest: installs from
- Marketplace manifest → Core plugin manifest: lists capability
- Core plugin manifest → Core skill: packages
- Core plugin manifest → Core MCP server: launches
- Core MCP server → MCP framework: uses
- Core MCP server → Media readers: exposes
- Core MCP server → Vision API adapters: exposes
- Core MCP server → Artifact renderers: exposes
- Media readers → Shared runtime services: uses
- Vision API adapters → Shared runtime services: uses
- Vision API adapters → DashScope & Serper: calls
- Artifact renderers → Local tools & desktop apps: invokes
- Memory build worker → Event graph builder: builds
- Event graph builder → Memory query service: persistent artifacts
- Memory build worker → Shared runtime services: uses APIs and media utilities
- Memory query service → MCP framework: serves via MCP
- Video editing tools → DashScope & Serper: generates media through
- Video editing tools → FFmpeg editing engine: uses execution plan
- FFmpeg editing engine → Local tools & desktop apps: requires FFmpeg
- Education renderer → Local tools & desktop apps: renders with local tools
- Blender thin client → Blender code tools: provides session to
- Blender code tools → Local tools & desktop apps: targets running Blender
- FreeCAD thin client → Local tools & desktop apps: targets running FreeCAD