How ksuriuri/index-tts-vllm works
IndexTTS-vLLM is a speech-synthesis application that replaces the IndexTTS GPT inference stage with vLLM for faster generation. A client submits text and optional reference audio through the API or web UI; inference prepares text and voice conditioning, generates acoustic tokens, converts them to audio features, and vocodes a waveform. The repository also documents v1/v1.5 multi-reference voice mixing and v2 emotion controls, plus OpenAI-compatible speech and voice-list APIs. The graph reflects documented capabilities and sampled inference/model code; some subsystem wiring is based on the README because its implementation was not sampled.
Main entry points: infer_vllm_v2.py v2 Inference · infer_vllm.py v1 Inference · api_server.py v1 API · api_server_v2.py v2 API
Components 22
Input Conditioning
- v1 Inference
indextts/infer_vllm.py - v2 Inference
indextts/infer_vllm_v2.py - Text Frontend
indextts/utils/front.py - Feature Extraction
indextts/utils/feature_extractors.py - Semantic Encoder
indextts/s2mel/wav2vecbert_extract.py - Voice Conditioning
indextts/s2mel/modules/campplus/DTDNN.py
Token Generation
- v1 vLLM GPT
indextts/gpt/model_vllm.py - v2 vLLM GPT
indextts/gpt/model_vllm_v2.py - v1 GPT Model
indextts/gpt/index_tts_gpt2_vllm_v1.py - Emotion Inference
indextts/infer_vllm_v2.py
Audio Synthesis
- S2Mel Synthesis
indextts/s2mel/modules/diffusion_transformer.py - Audio Features
indextts/s2mel/modules/audio.py - BigVGAN Vocoder
indextts/BigVGAN/bigvgan.py - BigVGAN Vocoder
indextts/s2mel/modules/bigvgan/bigvgan.py
Other components
- Client
- Browser User
- Generated Audio
- Voice List
Connections 30
- Client → v1 API: requests speech
- Client → v2 API: requests speech
- v1 API → v1 Inference: invokes
- v2 API → v2 Inference: invokes
- Browser User → v1 Web UI: submits speech
- Browser User → v2 Web UI: submits speech
- v1 Web UI → v1 Inference: invokes
- v2 Web UI → v2 Inference: invokes
- v1 API → Voice List: serves voices
- v2 API → Voice List: serves voices
- v1 Inference → Text Frontend: tokenizes text
- v2 Inference → Text Frontend: tokenizes text
- v1 Inference → Feature Extraction: extracts features
- v2 Inference → Feature Extraction: extracts features
- v1 Inference → Voice Conditioning: encodes voice
- v2 Inference → Voice Conditioning: encodes voice
- v2 Inference → Semantic Encoder: extracts semantics
- v1 Inference → v1 vLLM GPT: requests tokens
- v1 vLLM GPT → v1 GPT Model: runs model
- v2 Inference → v2 vLLM GPT: requests tokens
- v2 Inference → Emotion Inference: requests emotion
- v1 Inference → S2Mel Synthesis: converts tokens
- v2 Inference → S2Mel Synthesis: converts tokens
- S2Mel Synthesis → Audio Features: produces features
- v1 Inference → BigVGAN Vocoder: vocodes features
- v2 Inference → BigVGAN Vocoder: vocodes features
- BigVGAN Vocoder → Generated Audio: produces audio
- BigVGAN Vocoder → Generated Audio: produces audio
- v1 Inference → Generated Audio: returns audio
- v2 Inference → Generated Audio: returns audio