# modelscope/ms-swift architecture

> Architecture diagram of the GitHub repository modelscope/ms-swift, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-04-29). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/modelscope/ms-swift
- Repository: https://github.com/modelscope/ms-swift

## Overview

##### Project type and purpose
- SWIFT is an open-source Python framework and CLI tool for fine-tuning, evaluating, quantizing, and deploying large language and multi-modal models.
- It exposes both command-line commands (swift sft/infer/eval/export/... ) and a Python API to orchestrate model loading, training, inference engines, evaluation backends, quantization, and a Gradio-based Web UI.

##### File structure
- Top-level directories
- .dev_scripts, .github, Makefile, setup.py, requirements*: CI, build and packaging
- docs: Sphinx/ReadTheDocs documentation
- examples: dozens of shell & Python demos for training, inference, evaluation, deployment, quantization
- scripts: benchmark and utilities
- swift: the core library
- tests: unit/integration tests
- Inside swift/
- cli: command-line entrypoints (sft, pt, infer, eval, export, deploy, web_ui...)
- hub: model/dataset registry and download logic
- llm: high-level workflows for loading models, preprocessing, training, inference, export
- megatron: integration with Megatron-style parallel training
- plugin: hooks for custom losses, optimizers, metrics, templates, agent tools
- trainers: PyTorch/RLHF trainer abstractions, optimizer wrappers, callbacks
- tuners: lightweight fine-tuning methods (LoRA, QLoRA, LlamaPro…)
- ui: Gradio-based web interface components
- utils: logging, IO, environment, tensor utilities

##### README for architecture insights
- The README describes core features: CLI commands map to engine modules (pt, sft, rlhf, infer, eval, export, deploy, web-ui).
- It lists supported backends: PyTorch, DeepSpeed ZeRO, FSDP; inference engines: PyTorch, vLLM, LMDeploy; evaluation: EvalScope; quantization: GPTQ/AWQ/BNB.
- It highlights a plugin architecture for extensibility.

##### Final recommendations
- Major folder/module as a separate component.
- Plugin interfaces explicitly to reflect SWIFT’s extensibility.
- Do not overcrowd—group related modules under a “Core Library” cluster but break out engines and plugins.
- Technologies next to each component for clarity.

## Diagram (Mermaid)

```mermaid
flowchart TD
    %% CLI Layer
    subgraph "CLI Layer"
        Dispatcher["CLI Dispatcher"]:::cli
        Main["main.py"]:::cli
        Infer["infer.py"]:::cli
        SFT["sft.py"]:::cli
        RLHF["rlhf.py"]:::cli
        ExportCLI["export.py"]:::cli
        Deploy["deploy.py"]:::cli
        WebUI["web_ui.py"]:::cli
    end

    %% Core Library
    subgraph "Core Library"
        Hub["Model & Dataset Registry (Hub)"]:::core
        LLM["Core LLM Workflows"]:::core
        Trainers["Trainer Abstractions"]:::core
        Tuners["Tuner Modules"]:::core
        Plugin["Plugin Architecture"]:::plugin
        Megatron["Megatron Integration"]:::core
        UI["Web UI Components"]:::core
        Utils["Utilities"]:::core
    end

    %% External Dependencies
    subgraph "External Backends & Services"
        PyTorch["PyTorch/DeepSpeed/FSDP"]:::ext
        vLLM["vLLM"]:::ext
        LMDeploy["LMDeploy"]:::ext
        EvalScope["EvalScope"]:::ext
        ModelHub["ModelScope & HF Hub"]:::ext
        Gradio["Gradio"]:::ext
    end

    %% Connections
    Dispatcher --> Main
    Dispatcher --> Infer
    Dispatcher --> SFT
    Dispatcher --> RLHF
    Dispatcher --> ExportCLI
    Dispatcher --> Deploy
    Dispatcher --> WebUI

    Dispatcher --> Hub
    Dispatcher --> LLM
    Dispatcher --> Trainers
    Dispatcher --> Tuners
    Dispatcher --> Plugin
    Dispatcher --> Megatron
    Dispatcher --> UI
    Dispatcher --> Utils

    LLM --> Hub
    LLM --> Trainers
    LLM --> Tuners
    LLM --> Plugin
    LLM --> Megatron

    Trainers --> PyTorch
    Trainers --> PyTorch
    Trainers --> PyTorch

    LLM --> vLLM
    LLM --> LMDeploy
    LLM --> EvalScope

    Hub --> ModelHub
    UI --> Gradio

    %% Click Events
    click Main "https://github.com/modelscope/ms-swift/blob/main/swift/cli/main.py"
    click Infer "https://github.com/modelscope/ms-swift/blob/main/swift/cli/infer.py"
    click SFT "https://github.com/modelscope/ms-swift/blob/main/swift/cli/sft.py"
    click RLHF "https://github.com/modelscope/ms-swift/blob/main/swift/cli/rlhf.py"
    click ExportCLI "https://github.com/modelscope/ms-swift/blob/main/swift/cli/export.py"
    click Deploy "https://github.com/modelscope/ms-swift/blob/main/swift/cli/deploy.py"
    click WebUI "https://github.com/modelscope/ms-swift/blob/main/swift/cli/web_ui.py"
    click Hub "https://github.com/modelscope/ms-swift/tree/main/swift/hub/"
    click LLM "https://github.com/modelscope/ms-swift/tree/main/swift/llm/"
    click Trainers "https://github.com/modelscope/ms-swift/tree/main/swift/trainers/"
    click Tuners "https://github.com/modelscope/ms-swift/tree/main/swift/tuners/"
    click Plugin "https://github.com/modelscope/ms-swift/tree/main/swift/plugin/"
    click Megatron "https://github.com/modelscope/ms-swift/tree/main/swift/megatron/"
    click UI "https://github.com/modelscope/ms-swift/tree/main/swift/ui/"

    %% Styles
    classDef cli fill:#f9f,stroke:#333,stroke-width:1px;
    classDef core fill:#bbf,stroke:#333,stroke-width:1px;
    classDef ext fill:#bfb,stroke:#333,stroke-width:1px;
    classDef plugin fill:#ffd,stroke:#333,stroke-dasharray: 5 5;
```
