# modelscope/modelscope-agent architecture

> Architecture diagram of the GitHub repository modelscope/modelscope-agent, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-04-12). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/modelscope/modelscope-agent
- Repository: https://github.com/modelscope/modelscope-agent

## Overview

Based on our analysis of the file tree and the README content, this project is an open‐source, scalable agent framework for customizable, multi-agent, multi-task systems built on open-source large language models (LLMs). The framework integrates various agents (like RolePlay, CodexGraph, DataScience Assistant, Mobile Agent, etc.), multi-domain tools (such as code interpreter, weather query, drawing/image generation, web search, and more), and multiple LLM service providers (e.g., Dashscope, OpenAI, Zhipu). In short, it is a comprehensive system that orchestrates LLM interactions with a diverse set of tools and external APIs in order to build custom agent solutions.

##### Main Components
- Frontend & Admission: Although there is no “frontend” directory per se, the project supports interactive demos (Jupyter notebooks, streamlit, and web UIs in multi_roles_chat_room and Facechain Agent). So, you may want to represent an “Interactive Client/Demo UI” layer.
- Agent Core: Represent the base “Agent” module which is implemented by various agent types – for example, RolePlay agents in the “apps/agentfabric” directory, CodexGraph agent in “apps/codexgraph_agent”, DataScience assistant, Mobile Agent, etc.
- LLM Layer: Show the LLM component that is abstracted in models/llm (in modelscope_agent/llm) where multiple providers like Dashscope, Zhipu, OpenAI, and others are integrated.
- Tool Modules: Illustrate the “Tool” system (tools directory inside modelscope_agent/tools and similar directories) with various built-in tools (weather, code interpreter, web search, image generation, etc.). Emphasize the registration mechanism and integration with LLM for determining tool invocation.
- Memory, Storage & State Management: Include components for memory (e.g., modelscope_agent/memory) and storage (file and vector storages in modelscope_agent/storage) that help persist state and agent data.
- Multi-agent / Orchestration Components: Represent the multi-agent support (modelscope_agent/multi_agents_utils) and external orchestration services (modelscope_agent_servers/assistant_server, tool_manager_server, etc.). These components expose APIs for external interactions.
- External API Integrations: Show external services such as API calls to Dashscope, AMAP weather, external image or video generation services, and external model token services.
- Configuration & Deployment: Include supporting components like configuration files (in config/), docker scripts (in docker/), CI/CD configurations (in .github/workflows), and build scripts (Makefile, setup.py) that enable deployment and integration.

##### Relationships and Data Flow
- The workflow begins with a user/client request (in demo notebooks, streamlit pages, or API calls) reaching one of the agent applications.
- The Agent component (from apps or modelscope_agent core) receives the request and consults its LLM module with a specific “instruction” or role description.
- The LLM returns a response that may trigger tool invocations. At this stage, the system routes requests to the relevant Tool module(s) (e.g., weather lookup, image generation, code interpretation).
- Intermediate results or responses can be stored using Memory/Storage components.
- For multi-agent scenarios and scalability, the multi-agent utils and ModelScope-Agent-Server architectures interact to ensure parallel tool calling and orchestration.
- External APIs (like Dashscope for LLM calls or weather API) are invoked as part of tool execution.

##### Architectural Patterns and Design Principles
- Modular Extensibility: The tool registration and agent definitions are built for easy extension. Emphasize the plug-and-play design (e.g., show the registration mechanism in the modelscope_agent.tools and modelscope_agent.agents_registry).
- Microservice/Server Components: For deployment and production, the tool manager server, assistant server, and tool node server indicate that the system can be decomposed into separate service modules.
- Integration with External Services: LLM service providers and external APIs suggest a design that embraces external integrations with well-documented API interfaces.

##### Incorporate Specific Technologies and Frameworks
- Python 3.8+: The architecture should note dependency on Python and its ecosystem.
- Docker & CI/CD: Represent the Docker environments and GitHub workflows (CI, linting, publish) as part of the deployment pipeline.
- Notebook-based demos and interactive UIs: Acknowledge the example notebooks and streamlit demos.

##### Specific Elements
- Clear labels: Use component names such as “RolePlay Agent”, “CodexGraph Agent”, “LLM Service Providers”, “Tool Registry”, “Memory & Storage”, and “Multi-Agent Servers”.
- Blue blocks for core frameworks (LLM, Agent, Memory).
- Green blocks for tools and plugins.
- Where applicable, capture auxiliary components (CI/CD, Docker, config management) to demonstrate deployment and build pipelines.

## Diagram (Mermaid)

```mermaid
flowchart TD
    %% Presentation Layer
    subgraph "Presentation Layer"
        UI1["Multi Roles Chat Room (Web UI)"]:::frontend
        UI2["Demo Notebooks (Interactive)"]:::frontend
    end

    %% Business/Orchestration Layer
    subgraph "Business/Orchestration Layer"
        subgraph "Agent Core"
            A1["RolePlay Agent"]:::agent
            A2["CodexGraph Agent"]:::agent
            A3["DataScience Assistant"]:::agent
            A4["Mobile Agent"]:::agent
        end
        LLM["LLM Integration"]:::llm
        TOOL["Tool Modules"]:::tools
        subgraph "State Management"
            MEM["Memory"]:::memory
            STOR["Storage"]:::memory
        end
        subgraph "Multi-agent Orchestration"
            MAU["Multi-Agent Utils"]:::orchestration
            ASS["Assistant Server"]:::orchestration
            TMS["Tool Manager Server"]:::orchestration
            TNS["Tool Node Server"]:::orchestration
        end
    end

    %% Data & External Integration Layer
    subgraph "Data & External Integration Layer"
        subgraph "LLM Service Providers"
            DASH["Dashscope Provider"]:::external
            OPEN["OpenAI Provider"]:::external
            ZHIPU["Zhipu Provider"]:::external
        end
        subgraph "Tool External Integrations"
            WEATHER["Weather API"]:::external
            WEBSEARCH["Web Search Tool"]:::external
        end
    end

    %% Deployment & Config
    subgraph "Deployment & Config"
        CONF["Configuration Files"]:::config
        DOCKER["Docker Deployment"]:::config
        CICD["CI/CD Workflows"]:::config
        BUILD["Build & Deployment Scripts"]:::config
    end

    %% Connections: User Request flow
    UI1 -->|"userRequest"| A1
    UI2 -->|"userRequest"| A1

    %% Agent Core processing flow
    A1 -->|"instruction"| LLM
    A2 -->|"instruction"| LLM
    A3 -->|"instruction"| LLM
    A4 -->|"instruction"| LLM

    LLM -->|"LLMResponse"| TOOL

    TOOL -->|"storeResult"| MEM
    TOOL -->|"storeResult"| STOR

    TOOL -->|"triggerOrchestration"| MAU

    %% Multi-agent orchestration flow
    MAU -->|"orchestrates"| ASS
    MAU -->|"orchestrates"| TMS
    MAU -->|"orchestrates"| TNS

    %% External API integrations from LLM and Tools
    LLM -->|"callsLLM"| DASH
    LLM -->|"callsLLM"| OPEN
    LLM -->|"callsLLM"| ZHIPU

    TOOL -->|"invokes"| WEATHER
    TOOL -->|"invokes"| WEBSEARCH

    %% Aggregation of responses and state update
    MEM -->|"stateInfo"| A1
    STOR -->|"stateInfo"| A1
    ASS -->|"aggregates"| A1

    %% Final response back to UI
    A1 -->|"response"| UI1
    A1 -->|"response"| UI2

    %% Deployment & Config support
    CONF -->|"configures"| A1
    DOCKER -->|"containerizes"| A1
    CICD -->|"automates"| A1
    BUILD -->|"builds"| A1

    %% Click Events
    click UI1 "https://github.com/modelscope/modelscope-agent/tree/master/apps/multi_roles_chat_room"
    click UI2 "https://github.com/modelscope/modelscope-agent/tree/master/examples"
    
    click A1 "https://github.com/modelscope/modelscope-agent/tree/master/apps/agentfabric"
    click A2 "https://github.com/modelscope/modelscope-agent/tree/master/apps/codexgraph_agent"
    click A3 "https://github.com/modelscope/modelscope-agent/tree/master/apps/datascience_assistant"
    click A4 "https://github.com/modelscope/modelscope-agent/tree/master/apps/mobile_agent"
    
    click LLM "https://github.com/modelscope/modelscope-agent/tree/master/modelscope_agent/llm"
    
    click TOOL "https://github.com/modelscope/modelscope-agent/tree/master/modelscope_agent/tools"
    
    click MEM "https://github.com/modelscope/modelscope-agent/tree/master/modelscope_agent/memory"
    click STOR "https://github.com/modelscope/modelscope-agent/tree/master/modelscope_agent/storage"
    
    click MAU "https://github.com/modelscope/modelscope-agent/tree/master/modelscope_agent/multi_agents_utils"
    click ASS "https://github.com/modelscope/modelscope-agent/tree/master/modelscope_agent_servers/assistant_server"
    click TMS "https://github.com/modelscope/modelscope-agent/tree/master/modelscope_agent_servers/tool_manager_server"
    click TNS "https://github.com/modelscope/modelscope-agent/tree/master/modelscope_agent_servers/tool_node_server"
    
    click DASH "https://github.com/modelscope/modelscope-agent/blob/master/modelscope_agent/llm/dashscope.py"
    click OPEN "https://github.com/modelscope/modelscope-agent/blob/master/modelscope_agent/llm/openai.py"
    click ZHIPU "https://github.com/modelscope/modelscope-agent/blob/master/modelscope_agent/llm/zhipu.py"
    click WEATHER "https://github.com/modelscope/modelscope-agent/blob/master/modelscope_agent/tools/amap_weather.py"
    click WEBSEARCH "https://github.com/modelscope/modelscope-agent/tree/master/modelscope_agent/tools/web_search"
    
    click CONF "https://github.com/modelscope/modelscope-agent/tree/master/config"
    click DOCKER "https://github.com/modelscope/modelscope-agent/tree/master/docker"
    click CICD "https://github.com/modelscope/modelscope-agent/tree/master/.github/workflows"
    click BUILD "https://github.com/modelscope/modelscope-agent/tree/master/Makefile"

    %% Styles
    classDef frontend fill:#a7d8de,stroke:#2f7d8f,stroke-width:2px;
    classDef agent fill:#ffdbac,stroke:#e69500,stroke-width:2px;
    classDef llm fill:#d0f0c0,stroke:#6fa56f,stroke-width:2px;
    classDef tools fill:#f1c0e8,stroke:#d472b5,stroke-width:2px;
    classDef memory fill:#f9e79f,stroke:#f39c12,stroke-width:2px;
    classDef orchestration fill:#c3aed6,stroke:#8e44ad,stroke-width:2px;
    classDef external fill:#dcdcdc,stroke:#808080,stroke-width:2px,stroke-dasharray: 5,5;
    classDef config fill:#f5b7b1,stroke:#c0392b,stroke-width:2px;
```
