How sgl-project/sglang-omni works
SGLang-Omni is a multi-stage serving runtime for multimodal, speech, transcription, and text-to-speech models, with an OpenAI-compatible API and a multi-worker router. The principal workflow is a client request entering a serving surface, being scheduled across model-specific stages, transferring payloads between stages, and returning generated audio, text, or multimodal output. The graph groups model families rather than enumerating each implementation. Relationship evidence is limited to the README and sampled/indexed files; unsampled model wiring is not asserted.
Main entry points: speech_service.py OpenAI API · stage_workers.py Stage workers · engine_factory.py Engine factory · qwen3_asr Transcription models
Components 21
Serving surfaces
- OpenAI API
sglang_omni/serve/speech_service.py - Realtime sessions
sglang_omni/serve/realtime/manager.py - Omni router
sglang_omni_router/python/app.py
Pipeline runtime
- Request contract
sglang_omni/proto/request.py - Stage workers
sglang_omni/pipeline/stage_workers.py - Stage I/O
sglang_omni/comm/stage_io.py - Communication engine
sglang_omni/comm/engine.py - Pipeline state
sglang_omni/scheduling/pipeline_state.py - Engine factory
sglang_omni/scheduling/engine_factory.py - AR scheduler
sglang_omni/scheduling/omni_scheduler.py - Output processor
sglang_omni/scheduling/sglang_backend/output_processor.py - Model worker
sglang_omni/model_runner/model_worker.py - Runtime config
sglang_omni/config/manager.py
Model integrations
- Transcription models
sglang_omni/models/qwen3_asr - Speech models
sglang_omni/models/qwen3_tts - Omni models
sglang_omni/models/qwen3_omni - Music generation
sglang_omni/models/minimax_music3
Execution infrastructure
- Stage scheduling
sglang_omni/scheduling/engine_factory.py - Payload transport
sglang_omni/comm/engine.py
Other components
- API client
- SGLang
Connections 22
- API client → OpenAI API: submits request
- API client → Omni router: selects workers
- OpenAI API → Request contract: accepts requests
- OpenAI API → Stage workers: serves through
- API client → Realtime sessions: opens session
- Omni router → OpenAI API: routes requests
- Stage workers → Pipeline state: coordinates state
sglang_omni/pipeline/stage_workers.py - Stage workers → Communication engine: transfers payloads
sglang_omni/comm/engine.py - Stage workers → Engine factory: runs stages
- Engine factory → Model worker: builds worker
sglang_omni/scheduling/engine_factory.py - Engine factory → SGLang: composes execution
sglang_omni/scheduling/engine_factory.py - AR scheduler → Output processor: processes outputs
- Communication engine → Stage I/O: uses stage I/O
sglang_omni/comm/engine.py - Runtime config → Engine factory: resolves configuration
sglang_omni/config/manager.py - Transcription models → Engine factory: builds ASR stages
sglang_omni/models/qwen3_asr/engine_builder.py - Speech models → Engine factory: builds TTS stages
sglang_omni/models/qwen3_tts/engine_builder.py - Omni models → Stage workers: provides stages
- Music generation → Stage workers: provides stages
- Transcription models → OpenAI API: transcribes audio
- Speech models → OpenAI API: generates speech
- Omni models → OpenAI API: serves omni chat
- Music generation → OpenAI API: generates music