# sgl-project/sglang-omni architecture

> Architecture diagram of the GitHub repository sgl-project/sglang-omni, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-10-02). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/sgl-project/sglang-omni
- Repository: https://github.com/sgl-project/sglang-omni

## Overview

SGLang-Omni is a multi-stage serving runtime for multimodal, speech, transcription, and text-to-speech models, with an OpenAI-compatible API and a multi-worker router. The principal workflow is a client request entering a serving surface, being scheduled across model-specific stages, transferring payloads between stages, and returning generated audio, text, or multimodal output. The graph groups model families rather than enumerating each implementation. Relationship evidence is limited to the README and sampled/indexed files; unsampled model wiring is not asserted.

Main entry points: [`sglang_omni/serve/speech_service.py`](https://github.com/sgl-project/sglang-omni/blob/main/sglang_omni/serve/speech_service.py) (OpenAI API), [`sglang_omni/pipeline/stage_workers.py`](https://github.com/sgl-project/sglang-omni/blob/main/sglang_omni/pipeline/stage_workers.py) (Stage workers), [`sglang_omni/scheduling/engine_factory.py`](https://github.com/sgl-project/sglang-omni/blob/main/sglang_omni/scheduling/engine_factory.py) (Engine factory), [`sglang_omni/models/qwen3_asr`](https://github.com/sgl-project/sglang-omni/tree/main/sglang_omni/models/qwen3_asr) (Transcription models)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_entry["Serving surfaces"]
  node_openai_api["OpenAI API<br/>[speech_service.py]"]
  node_realtime["Realtime sessions<br/>[manager.py]"]
  node_router["Omni router<br/>[app.py]"]
end

subgraph group_runtime["Pipeline runtime"]
  node_request["Request contract<br/>[request.py]"]
  node_stage_workers["Stage workers<br/>[stage_workers.py]"]
  node_stage_io["Stage I/O<br/>[stage_io.py]"]
  node_comm_engine["Communication engine<br/>[engine.py]"]
  node_pipeline_state["Pipeline state<br/>[pipeline_state.py]"]
  node_engine_factory["Engine factory<br/>[engine_factory.py]"]
  node_scheduler["AR scheduler<br/>[omni_scheduler.py]"]
  node_output_processor["Output processor"]
  node_model_worker["Model worker<br/>[model_worker.py]"]
  node_config["Runtime config<br/>[manager.py]"]
end

subgraph group_models["Model integrations"]
  node_asr["Transcription models"]
  node_tts["Speech models"]
  node_omni_models["Omni models"]
  node_music["Music generation"]
end

subgraph group_infra["Execution infrastructure"]
  node_scheduling["Stage scheduling<br/>[engine_factory.py]"]
  node_transport["Payload transport<br/>[engine.py]"]
end

node_client(("API client"))
node_sglang["SGLang"]

node_client -->|"submits request"| node_openai_api
node_client -->|"selects workers"| node_router
node_openai_api -.->|"accepts requests"| node_request
node_openai_api -->|"serves through"| node_stage_workers
node_client -->|"opens session"| node_realtime
node_router -->|"routes requests"| node_openai_api
node_stage_workers -->|"coordinates state"| node_pipeline_state
node_stage_workers -->|"transfers payloads"| node_comm_engine
node_stage_workers -.->|"runs stages"| node_engine_factory
node_engine_factory -->|"builds worker"| node_model_worker
node_engine_factory -->|"composes execution"| node_sglang
node_scheduler -.->|"processes outputs"| node_output_processor
node_comm_engine -->|"uses stage I/O"| node_stage_io
node_config -->|"resolves configuration"| node_engine_factory
node_asr -->|"builds ASR stages"| node_engine_factory
node_tts -->|"builds TTS stages"| node_engine_factory
node_omni_models -.->|"provides stages"| node_stage_workers
node_music -.->|"provides stages"| node_stage_workers
node_asr -->|"transcribes audio"| node_openai_api
node_tts -->|"generates speech"| node_openai_api
node_omni_models -->|"serves omni chat"| node_openai_api
node_music -->|"generates music"| node_openai_api

click node_openai_api "https://github.com/sgl-project/sglang-omni/blob/main/sglang_omni/serve/speech_service.py"
click node_realtime "https://github.com/sgl-project/sglang-omni/blob/main/sglang_omni/serve/realtime/manager.py"
click node_router "https://github.com/sgl-project/sglang-omni/blob/main/sglang_omni_router/python/app.py"
click node_request "https://github.com/sgl-project/sglang-omni/blob/main/sglang_omni/proto/request.py"
click node_stage_workers "https://github.com/sgl-project/sglang-omni/blob/main/sglang_omni/pipeline/stage_workers.py"
click node_stage_io "https://github.com/sgl-project/sglang-omni/blob/main/sglang_omni/comm/stage_io.py"
click node_comm_engine "https://github.com/sgl-project/sglang-omni/blob/main/sglang_omni/comm/engine.py"
click node_pipeline_state "https://github.com/sgl-project/sglang-omni/blob/main/sglang_omni/scheduling/pipeline_state.py"
click node_engine_factory "https://github.com/sgl-project/sglang-omni/blob/main/sglang_omni/scheduling/engine_factory.py"
click node_scheduler "https://github.com/sgl-project/sglang-omni/blob/main/sglang_omni/scheduling/omni_scheduler.py"
click node_output_processor "https://github.com/sgl-project/sglang-omni/blob/main/sglang_omni/scheduling/sglang_backend/output_processor.py"
click node_model_worker "https://github.com/sgl-project/sglang-omni/blob/main/sglang_omni/model_runner/model_worker.py"
click node_config "https://github.com/sgl-project/sglang-omni/blob/main/sglang_omni/config/manager.py"
click node_asr "https://github.com/sgl-project/sglang-omni/tree/main/sglang_omni/models/qwen3_asr"
click node_tts "https://github.com/sgl-project/sglang-omni/tree/main/sglang_omni/models/qwen3_tts"
click node_omni_models "https://github.com/sgl-project/sglang-omni/tree/main/sglang_omni/models/qwen3_omni"
click node_music "https://github.com/sgl-project/sglang-omni/tree/main/sglang_omni/models/minimax_music3"
click node_scheduling "https://github.com/sgl-project/sglang-omni/blob/main/sglang_omni/scheduling/engine_factory.py"
click node_transport "https://github.com/sgl-project/sglang-omni/blob/main/sglang_omni/comm/engine.py"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_openai_api,node_realtime,node_router,node_client toneBlue
class node_request,node_stage_workers,node_stage_io,node_comm_engine,node_pipeline_state,node_engine_factory,node_scheduler,node_output_processor,node_model_worker,node_config toneAmber
class node_asr,node_tts,node_omni_models,node_music toneMint
class node_scheduling,node_transport toneRose
class node_sglang toneIndigo
```

## Components

### Serving surfaces

- [OpenAI API](https://github.com/sgl-project/sglang-omni/blob/main/sglang_omni/serve/speech_service.py): `sglang_omni/serve/speech_service.py`
- [Realtime sessions](https://github.com/sgl-project/sglang-omni/blob/main/sglang_omni/serve/realtime/manager.py): `sglang_omni/serve/realtime/manager.py`
- [Omni router](https://github.com/sgl-project/sglang-omni/blob/main/sglang_omni_router/python/app.py): `sglang_omni_router/python/app.py`

### Pipeline runtime

- [Request contract](https://github.com/sgl-project/sglang-omni/blob/main/sglang_omni/proto/request.py): `sglang_omni/proto/request.py`
- [Stage workers](https://github.com/sgl-project/sglang-omni/blob/main/sglang_omni/pipeline/stage_workers.py): `sglang_omni/pipeline/stage_workers.py`
- [Stage I/O](https://github.com/sgl-project/sglang-omni/blob/main/sglang_omni/comm/stage_io.py): `sglang_omni/comm/stage_io.py`
- [Communication engine](https://github.com/sgl-project/sglang-omni/blob/main/sglang_omni/comm/engine.py): `sglang_omni/comm/engine.py`
- [Pipeline state](https://github.com/sgl-project/sglang-omni/blob/main/sglang_omni/scheduling/pipeline_state.py): `sglang_omni/scheduling/pipeline_state.py`
- [Engine factory](https://github.com/sgl-project/sglang-omni/blob/main/sglang_omni/scheduling/engine_factory.py): `sglang_omni/scheduling/engine_factory.py`
- [AR scheduler](https://github.com/sgl-project/sglang-omni/blob/main/sglang_omni/scheduling/omni_scheduler.py): `sglang_omni/scheduling/omni_scheduler.py`
- [Output processor](https://github.com/sgl-project/sglang-omni/blob/main/sglang_omni/scheduling/sglang_backend/output_processor.py): `sglang_omni/scheduling/sglang_backend/output_processor.py`
- [Model worker](https://github.com/sgl-project/sglang-omni/blob/main/sglang_omni/model_runner/model_worker.py): `sglang_omni/model_runner/model_worker.py`
- [Runtime config](https://github.com/sgl-project/sglang-omni/blob/main/sglang_omni/config/manager.py): `sglang_omni/config/manager.py`

### Model integrations

- [Transcription models](https://github.com/sgl-project/sglang-omni/tree/main/sglang_omni/models/qwen3_asr): `sglang_omni/models/qwen3_asr`
- [Speech models](https://github.com/sgl-project/sglang-omni/tree/main/sglang_omni/models/qwen3_tts): `sglang_omni/models/qwen3_tts`
- [Omni models](https://github.com/sgl-project/sglang-omni/tree/main/sglang_omni/models/qwen3_omni): `sglang_omni/models/qwen3_omni`
- [Music generation](https://github.com/sgl-project/sglang-omni/tree/main/sglang_omni/models/minimax_music3): `sglang_omni/models/minimax_music3`

### Execution infrastructure

- [Stage scheduling](https://github.com/sgl-project/sglang-omni/blob/main/sglang_omni/scheduling/engine_factory.py): `sglang_omni/scheduling/engine_factory.py`
- [Payload transport](https://github.com/sgl-project/sglang-omni/blob/main/sglang_omni/comm/engine.py): `sglang_omni/comm/engine.py`

### Other components

- **API client**
- **SGLang**

## Connections

- API client → OpenAI API: submits request
- API client → Omni router: selects workers
- OpenAI API → Request contract: accepts requests
- OpenAI API → Stage workers: serves through
- API client → Realtime sessions: opens session
- Omni router → OpenAI API: routes requests
- Stage workers → Pipeline state: coordinates state (evidence: `sglang_omni/pipeline/stage_workers.py`)
- Stage workers → Communication engine: transfers payloads (evidence: `sglang_omni/comm/engine.py`)
- Stage workers → Engine factory: runs stages
- Engine factory → Model worker: builds worker (evidence: `sglang_omni/scheduling/engine_factory.py`)
- Engine factory → SGLang: composes execution (evidence: `sglang_omni/scheduling/engine_factory.py`)
- AR scheduler → Output processor: processes outputs
- Communication engine → Stage I/O: uses stage I/O (evidence: `sglang_omni/comm/engine.py`)
- Runtime config → Engine factory: resolves configuration (evidence: `sglang_omni/config/manager.py`)
- Transcription models → Engine factory: builds ASR stages (evidence: `sglang_omni/models/qwen3_asr/engine_builder.py`)
- Speech models → Engine factory: builds TTS stages (evidence: `sglang_omni/models/qwen3_tts/engine_builder.py`)
- Omni models → Stage workers: provides stages
- Music generation → Stage workers: provides stages
- Transcription models → OpenAI API: transcribes audio
- Speech models → OpenAI API: generates speech
- Omni models → OpenAI API: serves omni chat
- Music generation → OpenAI API: generates music
