How blaizzy/mlx-audio-swift works
MLX Audio Swift is a modular MLX-based audio SDK for Apple platforms. Its principal workflows are text-to-speech generation and audio-to-text transcription; the broader product also supports language identification, voice activity and speaker analysis, speech-to-speech processing, and audio codecs/tokenizers. Core utilities support model and audio workflows. The README documents the major capabilities, while sampled sources show CLI loading and inference examples; unsampled model internals and cross-module runtime calls are not inferred here.
Main entry points: TTSModel.swift TTS model API · MLXAudioSTT.swift STT model API · AudioUtils.swift Audio utilities · MLXAudioLID Language identification
Components 19
Audio Core
- Audio utilities
Sources/MLXAudioCore/AudioUtils.swift - DSP utilities
Sources/MLXAudioCore/DSP.swift - Generation types
Sources/MLXAudioCore/Generation/GenerationTypes.swift - Audio playback
Sources/MLXAudioCore/AudioPlayer.swift
Speech Synthesis
- TTS model API
Sources/MLXAudioTTS/TTSModel.swift - TTS generation
Sources/MLXAudioTTS/Generation.swift - TTS model families
Sources/MLXAudioTTS/Models - Text phonemization
Sources/MLXAudioG2P/G2P.swift
Speech Analysis
- STT model API
Sources/MLXAudioSTT/MLXAudioSTT.swift - STT model families
Sources/MLXAudioSTT/Models - Streaming inference
Sources/MLXAudioSTT/Streaming/StreamingInferenceSession.swift - Language identification
Sources/MLXAudioLID - Activity and diarization
Sources/MLXAudioVAD
Speech Processing
- Speech-to-speech API
Sources/MLXAudioSTS/STSModel.swift - Enhancement and separation
Sources/MLXAudioSTS/Models
Audio Codecs
- Codec model API
Sources/MLXAudioCodecs/AudioCodecModel.swift - Codec and tokenizers
Sources/MLXAudioCodecs
Other components
- App developer
- Model repository
Connections 22
- App developer → TTS model API: requests synthesis
- TTS model API → TTS generation: generates audio
- TTS model API → TTS model families: loads model
- TTS model API → Text phonemization: processes text
- TTS model API → Generation types: uses parameters
- TTS model API → Audio utilities: uses audio utilities
- App developer → STT model API: submits audio
- STT model API → STT model families: loads model
- STT model API → Streaming inference: streams inference
- STT model API → Audio utilities: loads audio
- App developer → Language identification: requests language ID
- Language identification → Audio utilities: loads audio
- App developer → Activity and diarization: requests speech analysis
- App developer → Speech-to-speech API: requests audio processing
- Speech-to-speech API → Enhancement and separation: runs task
- Speech-to-speech API → Audio utilities: loads audio
- App developer → Codec model API: requests codec operation
- Codec model API → Codec and tokenizers: selects codec
- Codec model API → Audio utilities: loads audio
- TTS model API → Model repository: loads pretrained weights
- STT model API → Model repository: loads pretrained weights
- Language identification → Model repository: loads pretrained weights