# orcnnn/monitoring architecture

> Architecture diagram of the GitHub repository orcnnn/monitoring, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-10-24). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/orcnnn/monitoring
- Repository: https://github.com/orcnnn/monitoring

## Overview

##### Project Type and Purpose
- SWIFT is an open-source Python framework and CLI tool (ms-swift) for the entire large language/multimodal model pipeline: data loading, pre-training, fine-tuning, human alignment (RLHF), inference, evaluation, quantization, deployment, and a Gradio-based Web-UI.
- It integrates with external compute backends (PyTorch, vLLM, LMDeploy), evaluation backends (EvalScope), model/dataset hubs (ModelScope), and hardware platforms (GPU, NPU, CPU).

##### High-Level Components
A. Command-Line Interface (“swift” CLI)
- Entry point: swift/cli/main.py
- Subcommands: sft, pt, rlhf, infer, eval, export, deploy, web-ui
- Parses arguments → dispatches to corresponding module’s “app.py” or handler
B. Core Python Library (“swift” Package)
- Dataset layer (swift/llm/dataset): data loaders, preprocessors, registry
- Model layer (swift/llm/model): model factory, patchers, support for 450+ models
- Template layer (swift/llm/template): prompt templates, registry
- Training layer (swift/llm/train + swift/trainers + swift/tuners):
- Pre-training (pt), fine-tuning (sft), RLHF trainers, plugin/extensibility
- Distributed strategies: DDP, ZeRO, FSDP, device_map
- Tuners: LoRA, QLoRA, LlamaPro, LongLoRA, and dozens of others
- Inference layer (swift/llm/infer): multiple engine implementations (PtEngine, vLLM, LMDeploy, VLLM, LMDeploy) behind a common “InferEngine” protocol
- Export/Quantization layer (swift/llm/export): GPTQ, AWQ, BNB, merging lora, Ollama export
- Evaluation layer (swift/llm/eval): EvalScope backend integration
- Deployment layer (swift/cli/deploy + swift/llm/infer/deploy): server startup scripts for client-server pattern
- Plugin layer (swift/plugin): hooks for custom loss, optimizer, metrics, callbacks
- Utilities (swift/utils): IO, logging, environment detection, tensor/torch wrappers
C. Web-UI Layer (swift/ui)
- Gradio-based interactive UIs for train, infer, eval, export
- Modules: llm_train, llm_infer, llm_eval, llm_export
D. Examples & Scripts
- examples/: concrete shell & Python demos showing CLI usage, notebook samples
- scripts/: benchmarking, reporting, dataset/model info

##### External Dependencies & Infrastructure
- Compute Backends: PyTorch, vLLM, LMDeploy
- Model/Dataset Hub: ModelScope (via swift/hub)
- Hardware: CPU, GPU (NVIDIA A100/V100/T4), Ascend NPU
- Evaluation: EvalScope
- Packaging/CI: Docker, GitHub Actions, pre-commit, setup.py

##### Relationships and Data Flows
- User invokes “swift <command>” → CLI parser → calls core library APIs
- Core library orchestrates:
- Dataset layer → yields TokenizedDataset
- Template layer → wraps data into prompt format
- Training layer → uses Trainer Factory to instantiate correct Trainer → trains model on hardware
- Inference layer → creates InferEngine → sends InferRequest → returns InferResponse
- Export layer → reads model artifacts → quantizes/merges → writes output
- Evaluation layer → sends test data → collects metrics → generates report
- Deployment layer → launches server (HTTP/gRPC) → clients connect
- Web-UI wraps the same core APIs behind a Gradio interface

##### Architectural Patterns & Principles
- Modular, Layered Architecture: clear separation of concerns by function (dataset, model, train, infer, export, eval, deploy)
- Plugin/Extension Points: registry patterns for models, datasets, templates, tuners, optimizers, callbacks
- Factory/Strategy Patterns: TrainerFactory, InferEngine interface, multiple backend implementations
- CLI-Library Duality: CLI commands are thin wrappers over Python API
- Declarative Configurations: JSON/YAML for ds_config, trainer, device_map, CI

## Diagram (Mermaid)

```mermaid
flowchart TB
    %% Users
    CLIUser([CLI User]):::user
    WebUIUser([Web UI User]):::user

    %% CLI and Web-UI
    CLI["CLI ('swift' CLI)"]:::pkg
    WebUI["Web-UI (Gradio)"]:::pkg

    CLIUser -->|"invoke 'swift <command>'"| CLI
    WebUIUser -->|"use Gradio UI"| WebUI

    CLI -->|"CLI args → Core API call"| CoreLib
    WebUI -->|"UI call → Core API"| CoreLib

    %% Core Library
    subgraph "Core Library"
        direction TB
        subgraph "Dataset Layer"
            DS["Dataset<br/>(swift/llm/dataset)"]:::pkg
        end
        subgraph "Model Layer"
            ML["Model<br/>(swift/llm/model)"]:::pkg
        end
        subgraph "Template Layer"
            TM["Template<br/>(swift/llm/template)"]:::pkg
        end
        subgraph "Training Layer"
            TR["Training<br/>(swift/llm/train + swift/trainers + swift/tuners)"]:::pkg
        end
        subgraph "Inference Layer"
            INF["Inference<br/>(swift/llm/infer)"]:::pkg
        end
        subgraph "Export Layer"
            EX["Export/Quant<br/>(swift/llm/export)"]:::pkg
        end
        subgraph "Evaluation Layer"
            EV["Evaluation<br/>(swift/llm/eval)"]:::pkg
        end
        subgraph "Deployment Layer"
            DP["Deployment<br/>(swift/llm/infer/deploy)"]:::pkg
        end
        subgraph "Plugin Layer"
            PL["Plugin<br/>(swift/plugin)"]:::pkg
        end
        subgraph "Utilities"
            UT["Utils<br/>(swift/utils)"]:::pkg
        end
        subgraph "Hub Integration"
            HB["Model/Dataset Hub<br/>(swift/hub)"]:::db
        end
    end

    %% External Dependencies & Infrastructure
    ComputeBackends(["Compute Backends<br/>(PyTorch, vLLM, LMDeploy)"]):::ext
    EvalScope(["EvalScope"]):::ext
    ModelScope(["ModelScope Hub"]):::db
    Hardware(["Hardware<br/>(GPU, CPU, NPU)"]):::ext
    Storage(["Disk Outputs & Logs"]):::db

    %% Relationships to externals
    TR -->|"runs on"| Hardware
    INF -->|"uses engine → backend"| ComputeBackends
    INF -->|"fetch models"| ModelScope
    EV -->|"sends data → collects metrics"| EvalScope
    EX -->|"writes artifacts"| Storage
    DS -->|"register/load"| HB
    PL -->|"hooks into"| TR
    PL -->|"hooks into"| INF
    PL -->|"hooks into"| EV

    %% Click Events
    click CLI "https://github.com/orcnnn/monitoring/blob/main/swift/cli/main.py"
    click DS "https://github.com/orcnnn/monitoring/tree/main/swift/llm/dataset/"
    click ML "https://github.com/orcnnn/monitoring/tree/main/swift/llm/model/"
    click TM "https://github.com/orcnnn/monitoring/tree/main/swift/llm/template/"
    click TR "https://github.com/orcnnn/monitoring/tree/main/swift/llm/train/"
    click TR "https://github.com/orcnnn/monitoring/blob/main/swift/trainers/trainer_factory.py"
    click TR "https://github.com/orcnnn/monitoring/tree/main/swift/tuners/"
    click INF "https://github.com/orcnnn/monitoring/blob/main/swift/llm/infer/infer.py"
    click INF "https://github.com/orcnnn/monitoring/blob/main/swift/llm/infer/infer_engine/pt_engine.py"
    click EX "https://github.com/orcnnn/monitoring/blob/main/swift/llm/export/export.py"
    click EV "https://github.com/orcnnn/monitoring/blob/main/swift/llm/eval/eval.py"
    click DP "https://github.com/orcnnn/monitoring/blob/main/swift/llm/infer/deploy.py"
    click PL "https://github.com/orcnnn/monitoring/tree/main/swift/plugin/"
    click UT "https://github.com/orcnnn/monitoring/tree/main/swift/utils/"
    click HB "https://github.com/orcnnn/monitoring/tree/main/swift/hub/"
    click WebUI "https://github.com/orcnnn/monitoring/blob/main/swift/ui/app.py"

    %% Styles
    classDef pkg fill:#ADD8E6,stroke:#000000,color:#000000
    classDef db fill:#90EE90,stroke:#000000,color:#000000
    classDef ext fill:#FFA500,stroke:#000000,color:#000000
    classDef user fill:#FFFFFF,stroke:#000000,color:#000000,shape:circle
```
