qwenlm/qwen-mm-plugins

Purpose and Delivery Model

Qwen-MM-Plugins is a Python-based marketplace of independently installable multimodal agent capabilities. A capability combines agent instructions (“skill”) with an optional stdio MCP server exposing tools. It targets multiple agent harnesses rather than providing a standalone web application. Main distribution/install entry points are install.sh, .claude-plugin/marketplace.json, and per-capability plugin manifests such as src/capabilities/core/.claude-plugin/plugin.json.

Runtime Entry Points and MCP Boundary

Python MCP servers are launched on demand, generally through each package’s module entry point, e.g. src/capabilities/core/qwen_mm_plugins_core/__main__.py and src/capabilities/video-memory/qwen_mm_plugins_video_memory/__main__.py. src/mcp_framework.py is the likely shared server/tool-registration abstraction; stdio response streaming is implemented for core at src/capabilities/core/qwen_mm_plugins_core/stdio_streaming.py. Skills define model-facing workflow and tool-use guidance independently of the executable server, e.g. src/capabilities/core/skill/SKILL.md.

Shared Platform Layer

Cross-capability infrastructure centralizes configuration, API access, execution, caching, and media utilities:

• Environment/config resolution: src/shared/env.py

• DashScope and OpenAI-compatible API clients: src/shared/api_dashscope.py, src/shared/api_openai.py

• Process, retry, cache, and content/image/video utilities: src/shared/syscmd.py, src/shared/retry.py, src/shared/video.py

Configuration prefers process environment and supports a shared user config file. External API boundaries include DashScope for perception/generation and Serper for web/search functions; native rendering and parsing additionally depend on local system tools.

Core Multimodal Processing

The core capability is the base perception and artifact-rendering layer. Readers ingest images and videos; API adapters provide OCR, ASR, visual grounding, segmentation, vision chat, and web retrieval. Producers create derivative views such as crops or bounding-box overlays. Renderers turn diverse source artifacts—PDF, Office, notebooks, HTML, LaTeX, SVG, geo data, and 3D models—into inspectable output.

Paths: src/capabilities/core/qwen_mm_plugins_core/readers/video.py, src/capabilities/core/qwen_mm_plugins_core/apis/vision_chat.py, src/capabilities/core/qwen_mm_plugins_core/renderers/.

Long-Video Memory Pipeline

Video memory creates and queries a hierarchical event graph for long-video QA. The build workflow coordinates extraction/analysis, embeddings, graph assembly, and merging; the runtime server loads this representation and exposes temporal, textual, summary, and subgraph queries. This is a persistent-data boundary: build artifacts are separate from request-time querying.

Paths: src/capabilities/video-memory/skill/script/build_memory/pipeline_worker.py, src/capabilities/video-memory/skill/script/build_memory/build_graph.py, src/capabilities/video-memory/qwen_mm_plugins_video_memory/query_memory.py.

Creative and Education Workflows

Video editing combines model-backed image/video/speech generation tools with skill-driven editing plans, FFmpeg-oriented execution guidance, media assets, and review gates. Educational content is skill-only: it guides image/problem analysis through scripting, narration, storyboard, component composition, rendering, and validation for Chinese explainer videos/pages.

Paths: src/capabilities/video-edit/qwen_mm_plugins_video_edit/tools/wan_t2v.py, src/capabilities/video-edit/skill/engines/ffmpeg-direct.md, src/capabilities/edu-agent/skill/scripts/_render.py.

Stateful 3D/CAD Thin Clients

Blender and FreeCAD capabilities are clients for already-running desktop applications, not embedded geometry engines. Their loaders/launchers establish the application boundary; MCP tools execute code and manipulate or inspect the current scene/document. Blender additionally integrates external asset catalogs and 3D generation services; FreeCAD supports parametric operations and FEM analysis.

Paths: src/capabilities/blender/qwen_mm_plugins_blender/_launch.py, src/capabilities/blender/qwen_mm_plugins_blender/tools/execute_blender_code.py, src/capabilities/freecad/qwen_mm_plugins_freecad/tools/execute_code_async.py.

Infrastructure Dependencies

The intended runtime is Python packaged via pyproject.toml and launched with uvx. Local dependencies materially affect capabilities: FFmpeg for audio/video; LibreOffice, Blender, TeX Live, and Chromium for selected render/visualization paths; running Blender/FreeCAD instances for their respective thin clients. Third-party API credentials gate DashScope-backed capabilities and Serper-backed web retrieval.