How modelscope/ms-swift works
Project type and purpose
• SWIFT is an open-source Python framework and CLI tool for fine-tuning, evaluating, quantizing, and deploying large language and multi-modal models.
• It exposes both command-line commands (swift sft/infer/eval/export/... ) and a Python API to orchestrate model loading, training, inference engines, evaluation backends, quantization, and a Gradio-based Web UI.
File structure
• Top-level directories
• .dev_scripts, .github, Makefile, setup.py, requirements*: CI, build and packaging
• docs: Sphinx/ReadTheDocs documentation
• examples: dozens of shell & Python demos for training, inference, evaluation, deployment, quantization
• scripts: benchmark and utilities
• swift: the core library
• tests: unit/integration tests
• Inside swift/
• cli: command-line entrypoints (sft, pt, infer, eval, export, deploy, web_ui...)
• hub: model/dataset registry and download logic
• llm: high-level workflows for loading models, preprocessing, training, inference, export
• megatron: integration with Megatron-style parallel training
• plugin: hooks for custom losses, optimizers, metrics, templates, agent tools
• trainers: PyTorch/RLHF trainer abstractions, optimizer wrappers, callbacks
• tuners: lightweight fine-tuning methods (LoRA, QLoRA, LlamaPro…)
• ui: Gradio-based web interface components
• utils: logging, IO, environment, tensor utilities
README for architecture insights
• The README describes core features: CLI commands map to engine modules (pt, sft, rlhf, infer, eval, export, deploy, web-ui).
• It lists supported backends: PyTorch, DeepSpeed ZeRO, FSDP; inference engines: PyTorch, vLLM, LMDeploy; evaluation: EvalScope; quantization: GPTQ/AWQ/BNB.
• It highlights a plugin architecture for extensibility.
Final recommendations
• Major folder/module as a separate component.
• Plugin interfaces explicitly to reflect SWIFT’s extensibility.
• Do not overcrowd—group related modules under a “Core Library” cluster but break out engines and plugins.
• Technologies next to each component for clarity.