# opensqz/minicpm-v-cookbook architecture

> Architecture diagram of the GitHub repository opensqz/minicpm-v-cookbook, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-03-31). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/opensqz/minicpm-v-cookbook
- Repository: https://github.com/opensqz/minicpm-v-cookbook

## Overview

1. This repository is a cookbook, not a single application. Its architecture is organized around “recipes” for the MiniCPM-V / MiniCPM-o model family: inference examples, fine-tuning pipelines, quantization workflows, deployment guides, and runnable demos.

2. The documentation system is the top-level integration layer. `docs/` contains the Sphinx site, with localized content under `docs/locales/zh_CN`, and the root `README.md` serves as a navigation index that points to all major subsystems and supported runtimes.

3. The inference subsystem lives in `inference/`. It is a set of standalone examples for core multimodal tasks: single-image QA, multi-image QA, video understanding, PDF/document parsing, OCR, grounding, speech-to-text, text-to-speech, and voice cloning. Sample media in `inference/assets/` is part of the data path for reproducible demos.

4. The fine-tuning subsystem is in `finetune/`. It includes dataset guidance, training scripts (`finetune.py`, `trainer.py`), DeepSpeed configs, and framework-specific notes for Transformers, LLaMA-Factory, SWIFT, and Align-anything. This layer connects prepared datasets to model adaptation workflows.

5. The quantization subsystem is in `quantization/`. It provides AWQ, BNB, and GGUF conversion scripts plus docs for producing smaller, deployable model artifacts. These outputs are intended to feed directly into the deployment recipes rather than a separate serving stack.

6. The deployment subsystem is split by serving runtime in `deployment/`: vLLM and SGLang for high-throughput GPU serving, llama.cpp for CPU/on-device use, and Ollama for simplified local deployment. These are the canonical backends referenced by the cookbook’s serving matrix.

7. The demo subsystem is where the repo becomes interactive. `demo/` contains multiple frontends and backend variants: a FastAPI-style web demo, an iOS demo, a Gradio demo, an OpenWebUI integration, and an “omni_stream” real-time web client/server pair. Each demo is a different UI/runtime packaging of the same model capabilities.

8. The most complex runtime architecture is in `demo/web_demo/WebRTC_Demo/`. It combines a Vue/Vite frontend (`o45-frontend`), a Python HTTP backend (`cpp_server` and `omini_backend_code`), nginx, and docker-compose. This stack is designed for real-time voice/video interaction, with LiveKit used for room/session coordination and WebRTC-style streaming.

9. Inside that web demo backend, the main boundaries are clear: request/API handling (`api/`, `voice_chat/routes.py`), session/token state (`voice_chat/entity/`), media/audio processing (`common/utils/`, `voice_chat/vad/`), and service coordination (`services/`, including heartbeat and inference service management). Redis is used for shared state, and ONNX VAD models are bundled for speech activity detection.

10. The frontend architecture is component-heavy and mode-driven. The Vue app under `o45-frontend/src/` is split into reusable components, hooks, device-aware routers, and view variants for mobile/tablet/web. It manages voice/video call states, model configuration, timbre selection, playback, responsiveness, and network diagnostics, with multiple versioned components showing an evolving real-time UX.

11. The `demo/web_demo/omni_stream/` area is a second real-time streaming stack with the same pattern: a Vue web client, a Python model server, VAD utilities, and bundled reference audio. It appears optimized for omni-modal streaming interactions, separate from the WebRTC demo but sharing the same model-serving philosophy.

12. Shared assets and configuration form the repository’s support layer: reference WAV files, example images/videos, ONNX VAD weights, nginx configs, docker-compose files, and environment files. These are what make the cookbook reproducible across local, containerized, and device-specific deployments.

13. Overall data flow is: user-facing demo or example → frontend/UI or script → API/service layer → model backend or external serving runtime → generated text/audio/video output. The repo’s main architectural concern is not a single codebase’s internal business logic, but the interoperability between model formats, serving engines, and multimodal interaction frontends.

Main entry points: [`deployment/README.md`](https://github.com/opensqz/minicpm-v-cookbook/blob/main/deployment/README.md) (Deploy), [`README.md`](https://github.com/opensqz/minicpm-v-cookbook/blob/main/README.md) (README), [`demo/README.md`](https://github.com/opensqz/minicpm-v-cookbook/blob/main/demo/README.md) (Demo), [`demo/web_demo/WebRTC_Demo/README.md`](https://github.com/opensqz/minicpm-v-cookbook/blob/main/demo/web_demo/WebRTC_Demo/README.md) (WebRTC demo)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_docs["Docs"]
  node_readme["README<br/>index / nav<br/>[README.md]"]
  node_site["Sphinx site<br/>docs site<br/>[index.rst]"]
  node_inf_docs["Inference docs<br/>recipe docs<br/>[transformers.md]"]
  node_ft_docs["Fine-tune docs<br/>recipe docs<br/>[fintune.md]"]
  node_q_docs["Quant docs<br/>recipe docs<br/>[awq.md]"]
  node_run_docs["Deploy docs<br/>recipe docs<br/>[llama.cpp.md]"]
end

subgraph group_inference["Inference"]
  node_inference_root["Inference<br/>example suite<br/>[README.md]"]
  node_single_image["Image QA<br/>vision example<br/>[single_image.md]"]
  node_multimodal_tasks["Multimodal tasks<br/>example cluster<br/>[multi_images.md]"]
  node_speech_tasks["Speech tasks<br/>audio example cluster<br/>[speech2text.md]"]
  node_sample_media[("Sample media<br/>demo assets")]
end

subgraph group_finetune["Fine-tune"]
  node_finetune_root["Fine-tune<br/>training suite<br/>[readme.md]"]
  node_dataset_guidance["Datasets<br/>data prep"]
  node_train_pipeline["Trainer<br/>train pipeline<br/>[finetune.py]"]
  node_framework_notes["Frameworks<br/>integration notes"]
  node_ds_configs["DeepSpeed<br/>training config"]
end

subgraph group_quantization["Quantization"]
  node_quant_root["Quantize<br/>conversion suite<br/>[README.md]"]
  node_awq["AWQ<br/>quant recipe"]
  node_bnb["BNB<br/>quant recipe"]
  node_gguf["GGUF<br/>quant recipe"]
end

subgraph group_deployment["Deployment"]
  node_deploy_root["Deploy<br/>serving matrix<br/>[README.md]"]
  node_vllm["vLLM<br/>gpu serving<br/>[README.md]"]
  node_sglang["SGLang<br/>gpu serving<br/>[README.md]"]
  node_llamacpp["llama.cpp<br/>local serving<br/>[README.md]"]
  node_ollama["Ollama<br/>local serving<br/>[README.md]"]
end

subgraph group_demo["Demos"]
  node_demo_root["Demo<br/>interactive suite<br/>[README.md]"]
  node_ios_demo["iOS demo<br/>mobile demo<br/>[ios.md]"]
  node_gradio_demo["Gradio demo<br/>web demo<br/>[README.md]"]
  node_openwebui_demo["OpenWebUI<br/>integration demo<br/>[README.md]"]
  node_omni_stream["Omni stream<br/>realtime stack<br/>[README.md]"]
  node_webrtc_demo["WebRTC demo<br/>realtime stack<br/>[README.md]"]
  node_frontend["Frontend<br/>vue app<br/>[App.vue]"]
  node_backend["Backend<br/>python api<br/>[main.py]"]
  node_cpp_server["CPP server<br/>python http"]
end

node_readme -->|"introduces"| node_site
node_site -->|"indexes"| node_inf_docs
node_site -->|"indexes"| node_ft_docs
node_site -->|"indexes"| node_q_docs
node_site -->|"indexes"| node_run_docs
node_readme -->|"links"| node_inference_root
node_readme -->|"links"| node_finetune_root
node_readme -->|"links"| node_quant_root
node_readme -->|"links"| node_deploy_root
node_readme -->|"links"| node_demo_root
node_inference_root -->|"covers"| node_single_image
node_inference_root -->|"covers"| node_multimodal_tasks
node_inference_root -->|"covers"| node_speech_tasks
node_inference_root -->|"uses"| node_sample_media
node_finetune_root -->|"starts with"| node_dataset_guidance
node_finetune_root -->|"executes"| node_train_pipeline
node_finetune_root -->|"adapts to"| node_framework_notes
node_finetune_root -->|"configures"| node_ds_configs
node_quant_root -->|"includes"| node_awq
node_quant_root -->|"includes"| node_bnb
node_quant_root -->|"includes"| node_gguf
node_quant_root -.->|"feeds"| node_deploy_root
node_deploy_root -->|"routes to"| node_vllm
node_deploy_root -->|"routes to"| node_sglang
node_deploy_root -->|"routes to"| node_llamacpp
node_deploy_root -->|"routes to"| node_ollama
node_demo_root -->|"includes"| node_ios_demo
node_demo_root -->|"includes"| node_gradio_demo
node_demo_root -->|"includes"| node_openwebui_demo
node_demo_root -->|"includes"| node_omni_stream
node_demo_root -->|"includes"| node_webrtc_demo
node_webrtc_demo -->|"hosts"| node_frontend
node_webrtc_demo -->|"hosts"| node_backend
node_webrtc_demo -->|"hosts"| node_cpp_server
node_frontend -.->|"calls"| node_backend
node_frontend -.->|"served by"| node_webrtc_demo
node_backend -.->|"coordinates"| node_cpp_server
node_backend -.->|"uses"| node_deploy_root
node_omni_stream -->|"pairs with"| node_frontend
node_omni_stream -->|"pairs with"| node_backend
node_webrtc_demo -.->|"uses"| node_sample_media

click node_readme "https://github.com/opensqz/minicpm-v-cookbook/blob/main/README.md"
click node_site "https://github.com/opensqz/minicpm-v-cookbook/blob/main/docs/source/index.rst"
click node_inf_docs "https://github.com/opensqz/minicpm-v-cookbook/blob/main/docs/source/inference/transformers.md"
click node_ft_docs "https://github.com/opensqz/minicpm-v-cookbook/blob/main/docs/source/finetune/fintune.md"
click node_q_docs "https://github.com/opensqz/minicpm-v-cookbook/blob/main/docs/source/quantization/awq.md"
click node_run_docs "https://github.com/opensqz/minicpm-v-cookbook/blob/main/docs/source/run_locally/llama.cpp.md"
click node_inference_root "https://github.com/opensqz/minicpm-v-cookbook/blob/main/inference/README.md"
click node_single_image "https://github.com/opensqz/minicpm-v-cookbook/blob/main/inference/single_image.md"
click node_multimodal_tasks "https://github.com/opensqz/minicpm-v-cookbook/blob/main/inference/multi_images.md"
click node_speech_tasks "https://github.com/opensqz/minicpm-v-cookbook/blob/main/inference/speech2text.md"
click node_sample_media "https://github.com/opensqz/minicpm-v-cookbook/tree/main/inference/assets"
click node_finetune_root "https://github.com/opensqz/minicpm-v-cookbook/blob/main/finetune/readme.md"
click node_dataset_guidance "https://github.com/opensqz/minicpm-v-cookbook/blob/main/finetune/dataset_guidance.md"
click node_train_pipeline "https://github.com/opensqz/minicpm-v-cookbook/blob/main/finetune/finetune.py"
click node_framework_notes "https://github.com/opensqz/minicpm-v-cookbook/blob/main/finetune/finetune_llamafactory.md"
click node_ds_configs "https://github.com/opensqz/minicpm-v-cookbook/blob/main/finetune/ds_config_zero2.json"
click node_quant_root "https://github.com/opensqz/minicpm-v-cookbook/blob/main/quantization/README.md"
click node_awq "https://github.com/opensqz/minicpm-v-cookbook/blob/main/quantization/awq/minicpm-v4_5_awq_quantize.py"
click node_bnb "https://github.com/opensqz/minicpm-v-cookbook/blob/main/quantization/bnb/minicpm-v4_5_bnb_quantize.py"
click node_gguf "https://github.com/opensqz/minicpm-v-cookbook/blob/main/quantization/gguf/minicpm-v4_gguf_quantize.md"
click node_deploy_root "https://github.com/opensqz/minicpm-v-cookbook/blob/main/deployment/README.md"
click node_vllm "https://github.com/opensqz/minicpm-v-cookbook/blob/main/deployment/vllm/README.md"
click node_sglang "https://github.com/opensqz/minicpm-v-cookbook/blob/main/deployment/sglang/README.md"
click node_llamacpp "https://github.com/opensqz/minicpm-v-cookbook/blob/main/deployment/llama.cpp/README.md"
click node_ollama "https://github.com/opensqz/minicpm-v-cookbook/blob/main/deployment/ollama/README.md"
click node_demo_root "https://github.com/opensqz/minicpm-v-cookbook/blob/main/demo/README.md"
click node_ios_demo "https://github.com/opensqz/minicpm-v-cookbook/blob/main/demo/ios_demo/ios.md"
click node_gradio_demo "https://github.com/opensqz/minicpm-v-cookbook/blob/main/demo/web_demo/gradio/README.md"
click node_openwebui_demo "https://github.com/opensqz/minicpm-v-cookbook/blob/main/demo/web_demo/openwebui/README.md"
click node_omni_stream "https://github.com/opensqz/minicpm-v-cookbook/blob/main/demo/web_demo/omni_stream/omni_web_client/README.md"
click node_webrtc_demo "https://github.com/opensqz/minicpm-v-cookbook/blob/main/demo/web_demo/WebRTC_Demo/README.md"
click node_frontend "https://github.com/opensqz/minicpm-v-cookbook/blob/main/demo/web_demo/WebRTC_Demo/o45-frontend/src/App.vue"
click node_backend "https://github.com/opensqz/minicpm-v-cookbook/blob/main/demo/web_demo/WebRTC_Demo/omini_backend_code/code/main.py"
click node_cpp_server "https://github.com/opensqz/minicpm-v-cookbook/blob/main/demo/web_demo/WebRTC_Demo/cpp_server/minicpmo_cpp_http_server.py"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_readme,node_site,node_inf_docs,node_ft_docs,node_q_docs,node_run_docs toneBlue
class node_inference_root,node_single_image,node_multimodal_tasks,node_speech_tasks,node_sample_media toneAmber
class node_finetune_root,node_dataset_guidance,node_train_pipeline,node_framework_notes,node_ds_configs toneMint
class node_quant_root,node_awq,node_bnb,node_gguf toneRose
class node_deploy_root,node_vllm,node_sglang,node_llamacpp,node_ollama toneIndigo
class node_demo_root,node_ios_demo,node_gradio_demo,node_openwebui_demo,node_omni_stream,node_webrtc_demo,node_frontend,node_backend,node_cpp_server toneTeal
```

## Components

### Docs

Top-level documentation and localized site content.

- [README](https://github.com/opensqz/minicpm-v-cookbook/blob/main/README.md): index / nav · Repository entry point and cookbook navigator. · `README.md`
- [Sphinx site](https://github.com/opensqz/minicpm-v-cookbook/blob/main/docs/source/index.rst): docs site · Primary documentation system and localized content. · `docs/source/index.rst`
- [Inference docs](https://github.com/opensqz/minicpm-v-cookbook/blob/main/docs/source/inference/transformers.md): recipe docs · Example pages for multimodal inference tasks. · `docs/source/inference/transformers.md`
- [Fine-tune docs](https://github.com/opensqz/minicpm-v-cookbook/blob/main/docs/source/finetune/fintune.md): recipe docs · Training and adaptation guidance for supported frameworks. · `docs/source/finetune/fintune.md`
- [Quant docs](https://github.com/opensqz/minicpm-v-cookbook/blob/main/docs/source/quantization/awq.md): recipe docs · Documentation for quantization workflows and formats. · `docs/source/quantization/awq.md`
- [Deploy docs](https://github.com/opensqz/minicpm-v-cookbook/blob/main/docs/source/run_locally/llama.cpp.md): recipe docs · Guides for runtime-specific local serving stacks. · `docs/source/run_locally/llama.cpp.md`

### Inference

Standalone multimodal usage recipes and sample media.

- [Inference](https://github.com/opensqz/minicpm-v-cookbook/blob/main/inference/README.md): example suite · Standalone multimodal inference recipes and assets. · `inference/README.md`
- [Image QA](https://github.com/opensqz/minicpm-v-cookbook/blob/main/inference/single_image.md): vision example · Single-image question answering and reasoning. · `inference/single_image.md`
- [Multimodal tasks](https://github.com/opensqz/minicpm-v-cookbook/blob/main/inference/multi_images.md): example cluster · Multi-image, video, OCR, PDF parsing, grounding, and scene text recognition. · `inference/multi_images.md`
- [Speech tasks](https://github.com/opensqz/minicpm-v-cookbook/blob/main/inference/speech2text.md): audio example cluster · Speech-to-text, text-to-speech, and voice cloning examples. · `inference/speech2text.md`
- [Sample media](https://github.com/opensqz/minicpm-v-cookbook/tree/main/inference/assets): demo assets · Reference audio/video/pdf files used by inference recipes. · `inference/assets`

### Fine-tune

Training pipelines, dataset guidance, and framework-specific notes.

- [Fine-tune](https://github.com/opensqz/minicpm-v-cookbook/blob/main/finetune/readme.md): training suite · Model adaptation workflows and training entry points. · `finetune/readme.md`
- [Datasets](https://github.com/opensqz/minicpm-v-cookbook/blob/main/finetune/dataset_guidance.md): data prep · Dataset guidance and preparation notes for training. · `finetune/dataset_guidance.md`
- [Trainer](https://github.com/opensqz/minicpm-v-cookbook/blob/main/finetune/finetune.py): train pipeline · Main fine-tuning scripts and training orchestration. · `finetune/finetune.py`
- [Frameworks](https://github.com/opensqz/minicpm-v-cookbook/blob/main/finetune/finetune_llamafactory.md): integration notes · Transformers, LLaMA-Factory, SWIFT, and Align-anything guidance. · `finetune/finetune_llamafactory.md`
- [DeepSpeed](https://github.com/opensqz/minicpm-v-cookbook/blob/main/finetune/ds_config_zero2.json): training config · Distributed training configuration files. · `finetune/ds_config_zero2.json`

### Quantization

Artifact conversion workflows for smaller deployable model variants.

- [Quantize](https://github.com/opensqz/minicpm-v-cookbook/blob/main/quantization/README.md): conversion suite · Quantization workflows for deployable artifacts. · `quantization/README.md`
- [AWQ](https://github.com/opensqz/minicpm-v-cookbook/blob/main/quantization/awq/minicpm-v4_5_awq_quantize.py): quant recipe · Activation-aware quantization scripts and notes. · `quantization/awq/minicpm-v4_5_awq_quantize.py`
- [BNB](https://github.com/opensqz/minicpm-v-cookbook/blob/main/quantization/bnb/minicpm-v4_5_bnb_quantize.py): quant recipe · Bitsandbytes quantization workflow. · `quantization/bnb/minicpm-v4_5_bnb_quantize.py`
- [GGUF](https://github.com/opensqz/minicpm-v-cookbook/blob/main/quantization/gguf/minicpm-v4_gguf_quantize.md): quant recipe · GGUF conversion workflow for local runtimes. · `quantization/gguf/minicpm-v4_gguf_quantize.md`

### Deployment

Serving backends by runtime and target environment.

- [Deploy](https://github.com/opensqz/minicpm-v-cookbook/blob/main/deployment/README.md): serving matrix · Canonical serving backends grouped by runtime. · `deployment/README.md`
- [vLLM](https://github.com/opensqz/minicpm-v-cookbook/blob/main/deployment/vllm/README.md): gpu serving · High-throughput GPU serving recipes. · `deployment/vllm/README.md`
- [SGLang](https://github.com/opensqz/minicpm-v-cookbook/blob/main/deployment/sglang/README.md): gpu serving · Alternative GPU serving backend and guides. · `deployment/sglang/README.md`
- [llama.cpp](https://github.com/opensqz/minicpm-v-cookbook/blob/main/deployment/llama.cpp/README.md): local serving · CPU and on-device deployment recipes. · `deployment/llama.cpp/README.md`
- [Ollama](https://github.com/opensqz/minicpm-v-cookbook/blob/main/deployment/ollama/README.md): local serving · Simplified local deployment guides. · `deployment/ollama/README.md`

### Demos

Interactive frontends and real-time runtime packaging.

- [Demo](https://github.com/opensqz/minicpm-v-cookbook/blob/main/demo/README.md): interactive suite · Interactive frontends and runtime-specific demo packaging. · `demo/README.md`
- [iOS demo](https://github.com/opensqz/minicpm-v-cookbook/blob/main/demo/ios_demo/ios.md): mobile demo · Native iOS packaging and usage notes. · `demo/ios_demo/ios.md`
- [Gradio demo](https://github.com/opensqz/minicpm-v-cookbook/blob/main/demo/web_demo/gradio/README.md): web demo · Gradio-based demo client/server variants. · `demo/web_demo/gradio/README.md`
- [OpenWebUI](https://github.com/opensqz/minicpm-v-cookbook/blob/main/demo/web_demo/openwebui/README.md): integration demo · OpenWebUI integration for conversational use. · `demo/web_demo/openwebui/README.md`
- [Omni stream](https://github.com/opensqz/minicpm-v-cookbook/blob/main/demo/web_demo/omni_stream/omni_web_client/README.md): realtime stack · Real-time streaming web client and server pair. · `demo/web_demo/omni_stream/omni_web_client/README.md`
- [WebRTC demo](https://github.com/opensqz/minicpm-v-cookbook/blob/main/demo/web_demo/WebRTC_Demo/README.md): realtime stack · Full voice/video interaction stack with frontend, backend, nginx, and compose. · `demo/web_demo/WebRTC_Demo/README.md`
- [Frontend](https://github.com/opensqz/minicpm-v-cookbook/blob/main/demo/web_demo/WebRTC_Demo/o45-frontend/src/App.vue): vue app · Device-aware Vue/Vite UI with reusable components and route variants. · `demo/web_demo/WebRTC_Demo/o45-frontend/src/App.vue`
- [Backend](https://github.com/opensqz/minicpm-v-cookbook/blob/main/demo/web_demo/WebRTC_Demo/omini_backend_code/code/main.py): python api · Python backend for API routes, session state, media processing, and service coordination. · `demo/web_demo/WebRTC_Demo/omini_backend_code/code/main.py`
- [CPP server](https://github.com/opensqz/minicpm-v-cookbook/blob/main/demo/web_demo/WebRTC_Demo/cpp_server/minicpmo_cpp_http_server.py): python http · Standalone HTTP server used in the WebRTC demo packaging. · `demo/web_demo/WebRTC_Demo/cpp_server/minicpmo_cpp_http_server.py`

## Connections

- README → Sphinx site: introduces · README points into the docs site and cookbook structure.
- Sphinx site → Inference docs: indexes · Documentation site includes inference recipes.
- Sphinx site → Fine-tune docs: indexes · Documentation site includes fine-tuning recipes.
- Sphinx site → Quant docs: indexes · Documentation site includes quantization recipes.
- Sphinx site → Deploy docs: indexes · Documentation site includes local deployment guides.
- README → Inference: links · Root README navigates to inference examples.
- README → Fine-tune: links · Root README navigates to fine-tuning workflows.
- README → Quantize: links · Root README navigates to quantization workflows.
- README → Deploy: links · Root README navigates to deployment backends.
- README → Demo: links · Root README navigates to demo packages.
- Inference → Image QA: covers · Inference suite includes single-image QA.
- Inference → Multimodal tasks: covers · Inference suite groups multi-image, video, OCR, PDF, and grounding tasks.
- Inference → Speech tasks: covers · Inference suite includes speech and voice recipes.
- Inference → Sample media: uses · Examples depend on bundled reference media.
- Fine-tune → Datasets: starts with · Training workflows begin with dataset preparation.
- Fine-tune → Trainer: executes · Fine-tuning is driven by training scripts.
- Fine-tune → Frameworks: adapts to · Fine-tuning supports multiple training frameworks.
- Fine-tune → DeepSpeed: configures · DeepSpeed configs support distributed training.
- Quantize → AWQ: includes · Quantization suite includes AWQ conversion.
- Quantize → BNB: includes · Quantization suite includes BNB conversion.
- Quantize → GGUF: includes · Quantization suite includes GGUF conversion.
- Quantize → Deploy: feeds · Quantized artifacts are intended for deployment backends.
- Deploy → vLLM: routes to · Deployment matrix includes vLLM serving.
- Deploy → SGLang: routes to · Deployment matrix includes SGLang serving.
- Deploy → llama.cpp: routes to · Deployment matrix includes llama.cpp serving.
- Deploy → Ollama: routes to · Deployment matrix includes Ollama serving.
- Demo → iOS demo: includes · Demo suite includes iOS packaging.
- Demo → Gradio demo: includes · Demo suite includes Gradio frontends.
- Demo → OpenWebUI: includes · Demo suite includes OpenWebUI integration.
- Demo → Omni stream: includes · Demo suite includes a real-time omni stream stack.
- Demo → WebRTC demo: includes · Demo suite includes the WebRTC real-time stack.
- WebRTC demo → Frontend: hosts · WebRTC demo bundles a Vue/Vite frontend.
- WebRTC demo → Backend: hosts · WebRTC demo bundles the Python backend.
- WebRTC demo → CPP server: hosts · WebRTC demo also includes a standalone HTTP server variant.
- Frontend → Backend: calls · UI communicates with backend APIs and session services.
- Frontend → WebRTC demo: served by · Frontend is packaged inside the real-time demo stack.
- Backend → CPP server: coordinates · Backend and HTTP server are alternative or cooperating runtime pieces.
- Backend → Deploy: uses · Backend delegates model execution to serving runtimes.
- Omni stream → Frontend: pairs with · Omni stream uses a web client frontend.
- Omni stream → Backend: pairs with · Omni stream has a Python model-serving backend.
- WebRTC demo → Sample media: uses · Real-time demo relies on bundled audio/media assets.
