# blaizzy/mlx-audio-swift architecture

> Architecture diagram of the GitHub repository blaizzy/mlx-audio-swift, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-09-29). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/blaizzy/mlx-audio-swift
- Repository: https://github.com/blaizzy/mlx-audio-swift

## Overview

MLX Audio Swift is a modular MLX-based audio SDK for Apple platforms. Its principal workflows are text-to-speech generation and audio-to-text transcription; the broader product also supports language identification, voice activity and speaker analysis, speech-to-speech processing, and audio codecs/tokenizers. Core utilities support model and audio workflows. The README documents the major capabilities, while sampled sources show CLI loading and inference examples; unsampled model internals and cross-module runtime calls are not inferred here.

Main entry points: [`Sources/MLXAudioTTS/TTSModel.swift`](https://github.com/blaizzy/mlx-audio-swift/blob/main/Sources/MLXAudioTTS/TTSModel.swift) (TTS model API), [`Sources/MLXAudioSTT/MLXAudioSTT.swift`](https://github.com/blaizzy/mlx-audio-swift/blob/main/Sources/MLXAudioSTT/MLXAudioSTT.swift) (STT model API), [`Sources/MLXAudioCore/AudioUtils.swift`](https://github.com/blaizzy/mlx-audio-swift/blob/main/Sources/MLXAudioCore/AudioUtils.swift) (Audio utilities), [`Sources/MLXAudioLID`](https://github.com/blaizzy/mlx-audio-swift/tree/main/Sources/MLXAudioLID) (Language identification)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_core["Audio Core"]
  node_core["Audio utilities<br/>[AudioUtils.swift]"]
  node_core_dsp["DSP utilities<br/>[DSP.swift]"]
  node_core_gen["Generation types"]
  node_core_play["Audio playback<br/>[AudioPlayer.swift]"]
end

subgraph group_synthesis["Speech Synthesis"]
  node_tts["TTS model API<br/>[TTSModel.swift]"]
  node_tts_generation["TTS generation<br/>[Generation.swift]"]
  node_tts_models["TTS model families"]
  node_g2p["Text phonemization<br/>[G2P.swift]"]
end

subgraph group_recognition["Speech Analysis"]
  node_stt["STT model API<br/>[MLXAudioSTT.swift]"]
  node_stt_models["STT model families"]
  node_stt_stream["Streaming inference"]
  node_lid["Language identification"]
  node_vad["Activity and diarization"]
end

subgraph group_transform["Speech Processing"]
  node_sts["Speech-to-speech API<br/>[STSModel.swift]"]
  node_sts_models["Enhancement and separation"]
end

subgraph group_codecs["Audio Codecs"]
  node_codec_api["Codec model API"]
  node_codec_models["Codec and tokenizers"]
end

node_user(("App developer"))
node_model_hub[("Model repository")]

node_user -->|"requests synthesis"| node_tts
node_tts -->|"generates audio"| node_tts_generation
node_tts -->|"loads model"| node_tts_models
node_tts -.->|"processes text"| node_g2p
node_tts -->|"uses parameters"| node_core_gen
node_tts -->|"uses audio utilities"| node_core
node_user -->|"submits audio"| node_stt
node_stt -->|"loads model"| node_stt_models
node_stt -->|"streams inference"| node_stt_stream
node_stt -->|"loads audio"| node_core
node_user -->|"requests language ID"| node_lid
node_lid -->|"loads audio"| node_core
node_user -->|"requests speech analysis"| node_vad
node_user -->|"requests audio processing"| node_sts
node_sts -->|"runs task"| node_sts_models
node_sts -->|"loads audio"| node_core
node_user -->|"requests codec operation"| node_codec_api
node_codec_api -->|"selects codec"| node_codec_models
node_codec_api -->|"loads audio"| node_core
node_tts -.->|"loads pretrained weights"| node_model_hub
node_stt -.->|"loads pretrained weights"| node_model_hub
node_lid -.->|"loads pretrained weights"| node_model_hub

click node_core "https://github.com/blaizzy/mlx-audio-swift/blob/main/Sources/MLXAudioCore/AudioUtils.swift"
click node_core_dsp "https://github.com/blaizzy/mlx-audio-swift/blob/main/Sources/MLXAudioCore/DSP.swift"
click node_core_gen "https://github.com/blaizzy/mlx-audio-swift/blob/main/Sources/MLXAudioCore/Generation/GenerationTypes.swift"
click node_core_play "https://github.com/blaizzy/mlx-audio-swift/blob/main/Sources/MLXAudioCore/AudioPlayer.swift"
click node_tts "https://github.com/blaizzy/mlx-audio-swift/blob/main/Sources/MLXAudioTTS/TTSModel.swift"
click node_tts_generation "https://github.com/blaizzy/mlx-audio-swift/blob/main/Sources/MLXAudioTTS/Generation.swift"
click node_tts_models "https://github.com/blaizzy/mlx-audio-swift/tree/main/Sources/MLXAudioTTS/Models"
click node_g2p "https://github.com/blaizzy/mlx-audio-swift/blob/main/Sources/MLXAudioG2P/G2P.swift"
click node_stt "https://github.com/blaizzy/mlx-audio-swift/blob/main/Sources/MLXAudioSTT/MLXAudioSTT.swift"
click node_stt_models "https://github.com/blaizzy/mlx-audio-swift/tree/main/Sources/MLXAudioSTT/Models"
click node_stt_stream "https://github.com/blaizzy/mlx-audio-swift/blob/main/Sources/MLXAudioSTT/Streaming/StreamingInferenceSession.swift"
click node_lid "https://github.com/blaizzy/mlx-audio-swift/tree/main/Sources/MLXAudioLID"
click node_vad "https://github.com/blaizzy/mlx-audio-swift/tree/main/Sources/MLXAudioVAD"
click node_sts "https://github.com/blaizzy/mlx-audio-swift/blob/main/Sources/MLXAudioSTS/STSModel.swift"
click node_sts_models "https://github.com/blaizzy/mlx-audio-swift/tree/main/Sources/MLXAudioSTS/Models"
click node_codec_api "https://github.com/blaizzy/mlx-audio-swift/blob/main/Sources/MLXAudioCodecs/AudioCodecModel.swift"
click node_codec_models "https://github.com/blaizzy/mlx-audio-swift/tree/main/Sources/MLXAudioCodecs"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_core,node_core_dsp,node_core_gen,node_core_play toneBlue
class node_tts,node_tts_generation,node_tts_models,node_g2p,node_model_hub toneAmber
class node_stt,node_stt_models,node_stt_stream,node_lid,node_vad toneMint
class node_sts,node_sts_models toneRose
class node_codec_api,node_codec_models,node_user toneIndigo
```

## Components

### Audio Core

- [Audio utilities](https://github.com/blaizzy/mlx-audio-swift/blob/main/Sources/MLXAudioCore/AudioUtils.swift): `Sources/MLXAudioCore/AudioUtils.swift`
- [DSP utilities](https://github.com/blaizzy/mlx-audio-swift/blob/main/Sources/MLXAudioCore/DSP.swift): `Sources/MLXAudioCore/DSP.swift`
- [Generation types](https://github.com/blaizzy/mlx-audio-swift/blob/main/Sources/MLXAudioCore/Generation/GenerationTypes.swift): `Sources/MLXAudioCore/Generation/GenerationTypes.swift`
- [Audio playback](https://github.com/blaizzy/mlx-audio-swift/blob/main/Sources/MLXAudioCore/AudioPlayer.swift): `Sources/MLXAudioCore/AudioPlayer.swift`

### Speech Synthesis

- [TTS model API](https://github.com/blaizzy/mlx-audio-swift/blob/main/Sources/MLXAudioTTS/TTSModel.swift): `Sources/MLXAudioTTS/TTSModel.swift`
- [TTS generation](https://github.com/blaizzy/mlx-audio-swift/blob/main/Sources/MLXAudioTTS/Generation.swift): `Sources/MLXAudioTTS/Generation.swift`
- [TTS model families](https://github.com/blaizzy/mlx-audio-swift/tree/main/Sources/MLXAudioTTS/Models): `Sources/MLXAudioTTS/Models`
- [Text phonemization](https://github.com/blaizzy/mlx-audio-swift/blob/main/Sources/MLXAudioG2P/G2P.swift): `Sources/MLXAudioG2P/G2P.swift`

### Speech Analysis

- [STT model API](https://github.com/blaizzy/mlx-audio-swift/blob/main/Sources/MLXAudioSTT/MLXAudioSTT.swift): `Sources/MLXAudioSTT/MLXAudioSTT.swift`
- [STT model families](https://github.com/blaizzy/mlx-audio-swift/tree/main/Sources/MLXAudioSTT/Models): `Sources/MLXAudioSTT/Models`
- [Streaming inference](https://github.com/blaizzy/mlx-audio-swift/blob/main/Sources/MLXAudioSTT/Streaming/StreamingInferenceSession.swift): `Sources/MLXAudioSTT/Streaming/StreamingInferenceSession.swift`
- [Language identification](https://github.com/blaizzy/mlx-audio-swift/tree/main/Sources/MLXAudioLID): `Sources/MLXAudioLID`
- [Activity and diarization](https://github.com/blaizzy/mlx-audio-swift/tree/main/Sources/MLXAudioVAD): `Sources/MLXAudioVAD`

### Speech Processing

- [Speech-to-speech API](https://github.com/blaizzy/mlx-audio-swift/blob/main/Sources/MLXAudioSTS/STSModel.swift): `Sources/MLXAudioSTS/STSModel.swift`
- [Enhancement and separation](https://github.com/blaizzy/mlx-audio-swift/tree/main/Sources/MLXAudioSTS/Models): `Sources/MLXAudioSTS/Models`

### Audio Codecs

- [Codec model API](https://github.com/blaizzy/mlx-audio-swift/blob/main/Sources/MLXAudioCodecs/AudioCodecModel.swift): `Sources/MLXAudioCodecs/AudioCodecModel.swift`
- [Codec and tokenizers](https://github.com/blaizzy/mlx-audio-swift/tree/main/Sources/MLXAudioCodecs): `Sources/MLXAudioCodecs`

### Other components

- **App developer**
- **Model repository**

## Connections

- App developer → TTS model API: requests synthesis
- TTS model API → TTS generation: generates audio
- TTS model API → TTS model families: loads model
- TTS model API → Text phonemization: processes text
- TTS model API → Generation types: uses parameters
- TTS model API → Audio utilities: uses audio utilities
- App developer → STT model API: submits audio
- STT model API → STT model families: loads model
- STT model API → Streaming inference: streams inference
- STT model API → Audio utilities: loads audio
- App developer → Language identification: requests language ID
- Language identification → Audio utilities: loads audio
- App developer → Activity and diarization: requests speech analysis
- App developer → Speech-to-speech API: requests audio processing
- Speech-to-speech API → Enhancement and separation: runs task
- Speech-to-speech API → Audio utilities: loads audio
- App developer → Codec model API: requests codec operation
- Codec model API → Codec and tokenizers: selects codec
- Codec model API → Audio utilities: loads audio
- TTS model API → Model repository: loads pretrained weights
- STT model API → Model repository: loads pretrained weights
- Language identification → Model repository: loads pretrained weights