# qwenlm/qwen3guard architecture

> Architecture diagram of the GitHub repository qwenlm/qwen3guard, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-06-25). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/qwenlm/qwen3guard
- Repository: https://github.com/qwenlm/qwen3guard

## Overview

1. This repository is primarily a **model release and evaluation repo**, not an application codebase. Its main purpose is to document Qwen3Guard, point users to model checkpoints, and provide scripts for benchmarking moderation behavior.

2. The core product boundary is the **Qwen3Guard model family** itself, released in two operational modes:
   - **Qwen3Guard-Gen**: a generative moderation model that reads a prompt or prompt+response pair and emits a structured safety verdict.
   - **Qwen3Guard-Stream**: a token-level streaming moderator that maintains state across incremental generation and classifies text in real time.

3. The repo’s **documentation layer** is centered in `README.md` and the attached technical report PDF. These define the safety policy, supported labels, example inputs/outputs, and the intended deployment patterns. The README is the main architecture reference for using the models correctly.

4. The **generative moderation flow** is:
   user prompt or user prompt + assistant response → Qwen chat template → transformer inference → structured text output → regex-like parsing by the caller into `Safety`, `Categories`, and optionally `Refusal`.
   This makes the output format part of the interface contract, not just free-form generation.

5. The **streaming moderation flow** is different:
   tokenized conversation → initial moderation on the user turn → repeated token-by-token calls with maintained `stream_state` → per-token risk/category updates → stream closure.
   This design is stateful and intended to sit alongside an LLM decoder, not replace it.

6. The repo clearly separates **offline/local inference** from **serving deployment**:
   - Local inference uses `transformers`.
   - Deployment for Qwen3Guard-Gen is shown through **vLLM** or **SGLang** as OpenAI-compatible endpoints.
   - The README notes that streaming support in those serving stacks is still future work for Qwen3Guard-Stream.

7. The `eval/` directory is the **benchmarking subsystem**. It contains:
   - `eval_gen.py` for evaluating generative moderation,
   - `eval_stream.py` for evaluating streaming moderation,
   - `eval/README.md` for benchmark-specific instructions.
   This separates evaluation logic from model usage examples.

8. Evaluation appears to be organized around the released **Qwen3GuardTest** benchmark, which focuses on moderation in streaming responses and reasoning-model outputs. The repo thus has a clear boundary between **model inference** and **benchmark scoring**.

9. The model’s **safety taxonomy** is part of the architecture because it drives both inference outputs and evaluation labels. It uses three severity levels:
   **Safe**, **Controversial**, and **Unsafe**,
   plus content categories such as Violent, PII, Suicide & Self-Harm, Copyright Violation, and Jailbreak for input-only detection.

10. The external distribution layer is important: checkpoints and datasets are hosted on **Hugging Face** and **ModelScope**. The repo itself does not store model weights; it acts as a pointer to those artifacts and as the documentation/evaluation entry point.

11. In architectural terms, the repo is a **thin control plane around hosted model artifacts**:
   - documentation and policy definition,
   - example inference flows,
   - serving guidance,
   - evaluation scripts.
   The actual learned moderation capability lives in the remote checkpoints, while this repo defines how to invoke and assess them.

Main entry points: [`README.md`](https://github.com/qwenlm/qwen3guard/blob/main/README.md) (README), [`eval/eval_stream.py`](https://github.com/qwenlm/qwen3guard/blob/main/eval/eval_stream.py) (eval_stream.py), [`eval/README.md`](https://github.com/qwenlm/qwen3guard/blob/main/eval/README.md) (Eval guide), [`eval/eval_gen.py`](https://github.com/qwenlm/qwen3guard/blob/main/eval/eval_gen.py) (eval_gen.py)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_docs["Documentation"]
  node_readme["README<br/>guide<br/>[README.md]"]
  node_report["Tech report<br/>paper"]
  node_safety_policy{{"Safety policy<br/>taxonomy<br/>[README.md]"}}
  node_release_refs[("Artifact links<br/>distribution<br/>[README.md]")]
end

subgraph group_models["Model Family"]
  node_qwen3guard[("Qwen3Guard<br/>model family")]
  node_gen_mode["Qwen3Guard-Gen<br/>generative model"]
  node_stream_mode["Qwen3Guard-Stream<br/>streaming model"]
  node_chat_template["Chat template<br/>prompting"]
  node_gen_infer["Gen inference<br/>transformers runtime"]
  node_gen_parse["Verdict parse<br/>output contract"]
  node_stream_state(("Stream state<br/>state machine"))
  node_stream_update["Token update<br/>streaming inference"]
  node_serve_vllm["vLLM serving<br/>openai endpoint"]
  node_serve_sglang["SGLang serving<br/>openai endpoint"]
end

subgraph group_eval["Evaluation"]
  node_eval_readme["Eval guide<br/>benchmark guide<br/>[README.md]"]
  node_eval_gen["eval_gen.py<br/>benchmark runner<br/>[eval_gen.py]"]
  node_eval_stream["eval_stream.py<br/>benchmark runner<br/>[eval_stream.py]"]
  node_qwen3guard_test[("Qwen3GuardTest<br/>benchmark set")]
end

node_readme -->|"defines"| node_safety_policy
node_readme -->|"points to"| node_release_refs
node_readme -->|"describes"| node_gen_mode
node_readme -->|"describes"| node_stream_mode
node_report -->|"covers"| node_qwen3guard
node_release_refs -->|"hosts"| node_qwen3guard
node_qwen3guard -->|"includes"| node_gen_mode
node_qwen3guard -->|"includes"| node_stream_mode
node_gen_mode -->|"uses"| node_chat_template
node_chat_template -->|"feeds"| node_gen_infer
node_gen_infer -->|"emits"| node_gen_parse
node_stream_mode -->|"maintains"| node_stream_state
node_stream_state -->|"updates"| node_stream_update
node_gen_mode -->|"deploys via"| node_serve_vllm
node_gen_mode -->|"deploys via"| node_serve_sglang
node_eval_readme -->|"guides"| node_eval_gen
node_eval_readme -->|"guides"| node_eval_stream
node_eval_gen -->|"scores"| node_gen_mode
node_eval_stream -->|"scores"| node_stream_mode
node_eval_stream -->|"uses"| node_qwen3guard_test

click node_readme "https://github.com/qwenlm/qwen3guard/blob/main/README.md"
click node_report "https://github.com/qwenlm/qwen3guard/blob/main/Qwen3Guard_Technical_Report.pdf"
click node_safety_policy "https://github.com/qwenlm/qwen3guard/blob/main/README.md"
click node_release_refs "https://github.com/qwenlm/qwen3guard/blob/main/README.md"
click node_eval_readme "https://github.com/qwenlm/qwen3guard/blob/main/eval/README.md"
click node_eval_gen "https://github.com/qwenlm/qwen3guard/blob/main/eval/eval_gen.py"
click node_eval_stream "https://github.com/qwenlm/qwen3guard/blob/main/eval/eval_stream.py"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_readme,node_report,node_safety_policy,node_release_refs toneBlue
class node_qwen3guard,node_gen_mode,node_stream_mode,node_chat_template,node_gen_infer,node_gen_parse,node_stream_state,node_stream_update,node_serve_vllm,node_serve_sglang toneAmber
class node_eval_readme,node_eval_gen,node_eval_stream,node_qwen3guard_test toneMint
```

## Components

### Documentation

Repo-facing policy, usage, and release guidance

- [README](https://github.com/qwenlm/qwen3guard/blob/main/README.md): guide · Main architecture and usage reference for Qwen3Guard · `README.md`
- [Tech report](https://github.com/qwenlm/qwen3guard/blob/main/Qwen3Guard_Technical_Report.pdf): paper · Technical report covering the model design, policy, and release details · `Qwen3Guard_Technical_Report.pdf`
- [Safety policy](https://github.com/qwenlm/qwen3guard/blob/main/README.md): taxonomy · Severity levels and content categories used by the moderation system · `README.md`
- [Artifact links](https://github.com/qwenlm/qwen3guard/blob/main/README.md): distribution · Pointers to hosted checkpoints and datasets on external model hubs · `README.md`

### Model Family

Released moderation checkpoints and runtime modes

- **Qwen3Guard**: model family · Hosted moderation checkpoints and associated runtime interfaces
- **Qwen3Guard-Gen**: generative model · Generates structured safety verdicts from prompts or prompt-response pairs
- **Qwen3Guard-Stream**: streaming model · Token-level moderator that maintains state across incremental generation
- **Chat template**: prompting · Formats inputs for generative inference before model execution
- **Gen inference**: transformers runtime · Local inference path for Qwen3Guard-Gen using transformer execution
- **Verdict parse**: output contract · Interprets generated text into Safety, Categories, and optional Refusal fields
- **Stream state**: state machine · Persistent moderation state carried across token-by-token updates
- **Token update**: streaming inference · Per-token risk and category updates during active generation
- **vLLM serving**: openai endpoint · Deployment path for Qwen3Guard-Gen via OpenAI-compatible serving
- **SGLang serving**: openai endpoint · Alternative deployment path for Qwen3Guard-Gen via OpenAI-compatible serving

### Evaluation

Benchmarking scripts and scoring entry points

- [Eval guide](https://github.com/qwenlm/qwen3guard/blob/main/eval/README.md): benchmark guide · Instructions and conventions for running benchmarks in eval/ · `eval/README.md`
- [eval_gen.py](https://github.com/qwenlm/qwen3guard/blob/main/eval/eval_gen.py): benchmark runner · Evaluation script for generative moderation behavior · `eval/eval_gen.py`
- [eval_stream.py](https://github.com/qwenlm/qwen3guard/blob/main/eval/eval_stream.py): benchmark runner · Evaluation script for streaming moderation behavior · `eval/eval_stream.py`
- **Qwen3GuardTest**: benchmark set · Released benchmark focused on streaming responses and reasoning outputs

## Connections

- README → Safety policy: defines · Documents the moderation labels, severity levels, and categories
- README → Artifact links: points to · Links users to external checkpoints and datasets
- README → Qwen3Guard-Gen: describes · Explains the generative moderation workflow and expected outputs
- README → Qwen3Guard-Stream: describes · Explains the stateful streaming moderation workflow
- Tech report → Qwen3Guard: covers · Technical background for the released model family
- Artifact links → Qwen3Guard: hosts · External artifact locations for the checkpoints
- Qwen3Guard → Qwen3Guard-Gen: includes · Generative moderation as one operational mode
- Qwen3Guard → Qwen3Guard-Stream: includes · Streaming moderation as one operational mode
- Qwen3Guard-Gen → Chat template: uses · Inputs are formatted with the Qwen chat template before inference
- Chat template → Gen inference: feeds · Prepared prompts are passed into local transformer inference
- Gen inference → Verdict parse: emits · Structured text is parsed into moderation fields
- Qwen3Guard-Stream → Stream state: maintains · Carries state across incremental calls
- Stream state → Token update: updates · State is refined by token-by-token moderation events
- Qwen3Guard-Gen → vLLM serving: deploys via · Shown as an OpenAI-compatible serving option
- Qwen3Guard-Gen → SGLang serving: deploys via · Shown as an OpenAI-compatible serving option
- Eval guide → eval_gen.py: guides · Benchmark instructions for generative evaluation
- Eval guide → eval_stream.py: guides · Benchmark instructions for streaming evaluation
- eval_gen.py → Qwen3Guard-Gen: scores · Measures generative moderation behavior
- eval_stream.py → Qwen3Guard-Stream: scores · Measures streaming moderation behavior
- eval_stream.py → Qwen3GuardTest: uses · Benchmark target for streaming and reasoning-response cases