# fluidinference/fluidaudio architecture

> Architecture diagram of the GitHub repository fluidinference/fluidaudio, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-09-25). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/fluidinference/fluidaudio
- Repository: https://github.com/fluidinference/fluidaudio

## Overview

FluidAudio is a Swift SDK for local audio AI on Apple devices. Applications supply audio to speech-recognition, voice-activity, and diarization APIs, or text to speech-synthesis APIs; results are transcripts, speech regions, speaker-labelled segments, embeddings, or generated audio. Its main ASR paths include batch/sliding-window and streaming variants, with additional language-specific managers. Shared audio/model utilities support these capabilities, while model inference uses Core ML and the Apple Neural Engine. README capabilities are represented at subsystem level; sampled excerpts show selected pipeline details, not every manager’s wiring.

Main entry points: [`Sources/FluidAudio/Shared/AudioSourceFactory.swift`](https://github.com/fluidinference/fluidaudio/blob/main/Sources/FluidAudio/Shared/AudioSourceFactory.swift) (Audio sources), [`Sources/FluidAudioCLI/FluidAudioCLI.swift`](https://github.com/fluidinference/fluidaudio/blob/main/Sources/FluidAudioCLI/FluidAudioCLI.swift) (FluidAudio CLI), [`Sources/FluidAudio/Diarizer/Core/DiarizerManager.swift`](https://github.com/fluidinference/fluidaudio/blob/main/Sources/FluidAudio/Diarizer/Core/DiarizerManager.swift) (Online diarization), [`Sources/FluidAudio/ASR/Parakeet/SlidingWindow/SlidingWindowAsrManager.swift`](https://github.com/fluidinference/fluidaudio/blob/main/Sources/FluidAudio/ASR/Parakeet/SlidingWindow/SlidingWindowAsrManager.swift) (ASR managers)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_shared["Shared runtime"]
  node_audio["Audio sources"]
  node_models["Model registry"]
  node_download["Model downloads<br/>[ModelHub.swift]"]
end

subgraph group_asr["Speech recognition"]
  node_asr["ASR managers"]
  node_tdt["TDT pipeline"]
  node_nemotron["Streaming ASR"]
  node_cohere["Cohere ASR"]
  node_paraformer["Paraformer ASR"]
  node_itn["Text normalization"]
end

subgraph group_analysis["Speech analysis"]
  node_vad["Voice activity<br/>[VadManager.swift]"]
  node_diarize["Online diarization"]
  node_speakers["Speaker tracking"]
  node_offline["Offline diarization"]
  node_embeddings["Speaker embeddings"]
end

subgraph group_tts["Speech synthesis"]
  node_kokoro["Kokoro TTS"]
  node_pocket["Pocket TTS"]
  node_pocketstate["Synthesis state"]
  node_ttsmodels["TTS managers"]
end

subgraph group_cli["Command line"]
  node_cli["FluidAudio CLI"]
end

node_app(("Application"))
node_coreml["Core ML / ANE"]

node_app -->|"provides audio"| node_audio
node_app -->|"requests transcription"| node_asr
node_app -->|"requests detection"| node_vad
node_app -->|"requests diarization"| node_diarize
node_app -->|"requests batch analysis"| node_offline
node_app -->|"requests embeddings"| node_embeddings
node_app -->|"provides text"| node_kokoro
node_app -->|"provides text"| node_pocket
node_app -->|"requests synthesis"| node_ttsmodels
node_audio -->|"supplies audio"| node_asr
node_audio -->|"supplies audio"| node_vad
node_audio -->|"supplies audio"| node_diarize
node_audio -->|"supplies audio"| node_offline
node_models -->|"obtains assets"| node_download
node_download -->|"returns assets"| node_models
node_asr -->|"runs windows"| node_tdt
node_tdt -->|"runs inference"| node_coreml
node_nemotron -->|"runs inference"| node_coreml
node_cohere -->|"runs inference"| node_coreml
node_paraformer -->|"runs inference"| node_coreml
node_vad -->|"runs inference"| node_coreml
node_diarize -->|"tracks speakers"| node_speakers
node_diarize -->|"runs inference"| node_coreml
node_offline -->|"runs inference"| node_coreml
node_embeddings -->|"runs inference"| node_coreml
node_kokoro -->|"runs inference"| node_coreml
node_pocket -->|"uses state"| node_pocketstate
node_pocketstate -->|"runs inference"| node_coreml
node_ttsmodels -->|"runs inference"| node_coreml
node_cli -->|"dispatches commands"| node_asr
node_cli -->|"dispatches commands"| node_vad
node_cli -->|"dispatches commands"| node_offline
node_cli -->|"dispatches commands"| node_ttsmodels

click node_audio "https://github.com/fluidinference/fluidaudio/blob/main/Sources/FluidAudio/Shared/AudioSourceFactory.swift"
click node_models "https://github.com/fluidinference/fluidaudio/blob/main/Sources/FluidAudio/ModelRegistry.swift"
click node_download "https://github.com/fluidinference/fluidaudio/blob/main/Sources/FluidAudio/Shared/Download/ModelHub.swift"
click node_asr "https://github.com/fluidinference/fluidaudio/blob/main/Sources/FluidAudio/ASR/Parakeet/SlidingWindow/SlidingWindowAsrManager.swift"
click node_tdt "https://github.com/fluidinference/fluidaudio/blob/main/Sources/FluidAudio/ASR/Parakeet/SlidingWindow/TDT/AsrManager%2BPipeline.swift"
click node_nemotron "https://github.com/fluidinference/fluidaudio/blob/main/Sources/FluidAudio/ASR/Parakeet/Streaming/Nemotron/StreamingNemotronAsrManager%2BPipeline.swift"
click node_cohere "https://github.com/fluidinference/fluidaudio/blob/main/Sources/FluidAudio/ASR/Cohere/CoherePipeline.swift"
click node_paraformer "https://github.com/fluidinference/fluidaudio/blob/main/Sources/FluidAudio/ASR/Paraformer/ParaformerManager.swift"
click node_itn "https://github.com/fluidinference/fluidaudio/blob/main/Sources/FluidAudio/ITN/TextNormalizer.swift"
click node_vad "https://github.com/fluidinference/fluidaudio/blob/main/Sources/FluidAudio/VAD/VadManager.swift"
click node_diarize "https://github.com/fluidinference/fluidaudio/blob/main/Sources/FluidAudio/Diarizer/Core/DiarizerManager.swift"
click node_speakers "https://github.com/fluidinference/fluidaudio/blob/main/Sources/FluidAudio/Diarizer/Clustering/SpeakerManager.swift"
click node_offline "https://github.com/fluidinference/fluidaudio/blob/main/Sources/FluidAudio/Diarizer/Offline/Core/OfflineDiarizerManager.swift"
click node_embeddings "https://github.com/fluidinference/fluidaudio/blob/main/Sources/FluidAudio/Speaker/CampPlusEmbedder.swift"
click node_kokoro "https://github.com/fluidinference/fluidaudio/blob/main/Sources/FluidAudio/TTS/KokoroAne/KokoroAneManager.swift"
click node_pocket "https://github.com/fluidinference/fluidaudio/blob/main/Sources/FluidAudio/TTS/PocketTTS/PocketTtsManager.swift"
click node_pocketstate "https://github.com/fluidinference/fluidaudio/blob/main/Sources/FluidAudio/TTS/PocketTTS/Pipeline/PocketTtsStateEngine.swift"
click node_ttsmodels "https://github.com/fluidinference/fluidaudio/blob/main/Sources/FluidAudio/TTS/Chatterbox/ChatterboxManager.swift"
click node_cli "https://github.com/fluidinference/fluidaudio/blob/main/Sources/FluidAudioCLI/FluidAudioCLI.swift"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_audio,node_models,node_download toneBlue
class node_asr,node_tdt,node_nemotron,node_cohere,node_paraformer,node_itn toneAmber
class node_vad,node_diarize,node_speakers,node_offline,node_embeddings toneMint
class node_kokoro,node_pocket,node_pocketstate,node_ttsmodels toneRose
class node_cli,node_app,node_coreml toneIndigo
```

## Components

### Shared runtime

- [Audio sources](https://github.com/fluidinference/fluidaudio/blob/main/Sources/FluidAudio/Shared/AudioSourceFactory.swift): `Sources/FluidAudio/Shared/AudioSourceFactory.swift`
- [Model registry](https://github.com/fluidinference/fluidaudio/blob/main/Sources/FluidAudio/ModelRegistry.swift): `Sources/FluidAudio/ModelRegistry.swift`
- [Model downloads](https://github.com/fluidinference/fluidaudio/blob/main/Sources/FluidAudio/Shared/Download/ModelHub.swift): `Sources/FluidAudio/Shared/Download/ModelHub.swift`

### Speech recognition

- [ASR managers](https://github.com/fluidinference/fluidaudio/blob/main/Sources/FluidAudio/ASR/Parakeet/SlidingWindow/SlidingWindowAsrManager.swift): `Sources/FluidAudio/ASR/Parakeet/SlidingWindow/SlidingWindowAsrManager.swift`
- [TDT pipeline](https://github.com/fluidinference/fluidaudio/blob/main/Sources/FluidAudio/ASR/Parakeet/SlidingWindow/TDT/AsrManager%2BPipeline.swift): `Sources/FluidAudio/ASR/Parakeet/SlidingWindow/TDT/AsrManager+Pipeline.swift`
- [Streaming ASR](https://github.com/fluidinference/fluidaudio/blob/main/Sources/FluidAudio/ASR/Parakeet/Streaming/Nemotron/StreamingNemotronAsrManager%2BPipeline.swift): `Sources/FluidAudio/ASR/Parakeet/Streaming/Nemotron/StreamingNemotronAsrManager+Pipeline.swift`
- [Cohere ASR](https://github.com/fluidinference/fluidaudio/blob/main/Sources/FluidAudio/ASR/Cohere/CoherePipeline.swift): `Sources/FluidAudio/ASR/Cohere/CoherePipeline.swift`
- [Paraformer ASR](https://github.com/fluidinference/fluidaudio/blob/main/Sources/FluidAudio/ASR/Paraformer/ParaformerManager.swift): `Sources/FluidAudio/ASR/Paraformer/ParaformerManager.swift`
- [Text normalization](https://github.com/fluidinference/fluidaudio/blob/main/Sources/FluidAudio/ITN/TextNormalizer.swift): `Sources/FluidAudio/ITN/TextNormalizer.swift`

### Speech analysis

- [Voice activity](https://github.com/fluidinference/fluidaudio/blob/main/Sources/FluidAudio/VAD/VadManager.swift): `Sources/FluidAudio/VAD/VadManager.swift`
- [Online diarization](https://github.com/fluidinference/fluidaudio/blob/main/Sources/FluidAudio/Diarizer/Core/DiarizerManager.swift): `Sources/FluidAudio/Diarizer/Core/DiarizerManager.swift`
- [Speaker tracking](https://github.com/fluidinference/fluidaudio/blob/main/Sources/FluidAudio/Diarizer/Clustering/SpeakerManager.swift): `Sources/FluidAudio/Diarizer/Clustering/SpeakerManager.swift`
- [Offline diarization](https://github.com/fluidinference/fluidaudio/blob/main/Sources/FluidAudio/Diarizer/Offline/Core/OfflineDiarizerManager.swift): `Sources/FluidAudio/Diarizer/Offline/Core/OfflineDiarizerManager.swift`
- [Speaker embeddings](https://github.com/fluidinference/fluidaudio/blob/main/Sources/FluidAudio/Speaker/CampPlusEmbedder.swift): `Sources/FluidAudio/Speaker/CampPlusEmbedder.swift`

### Speech synthesis

- [Kokoro TTS](https://github.com/fluidinference/fluidaudio/blob/main/Sources/FluidAudio/TTS/KokoroAne/KokoroAneManager.swift): `Sources/FluidAudio/TTS/KokoroAne/KokoroAneManager.swift`
- [Pocket TTS](https://github.com/fluidinference/fluidaudio/blob/main/Sources/FluidAudio/TTS/PocketTTS/PocketTtsManager.swift): `Sources/FluidAudio/TTS/PocketTTS/PocketTtsManager.swift`
- [Synthesis state](https://github.com/fluidinference/fluidaudio/blob/main/Sources/FluidAudio/TTS/PocketTTS/Pipeline/PocketTtsStateEngine.swift): `Sources/FluidAudio/TTS/PocketTTS/Pipeline/PocketTtsStateEngine.swift`
- [TTS managers](https://github.com/fluidinference/fluidaudio/blob/main/Sources/FluidAudio/TTS/Chatterbox/ChatterboxManager.swift): `Sources/FluidAudio/TTS/Chatterbox/ChatterboxManager.swift`

### Command line

- [FluidAudio CLI](https://github.com/fluidinference/fluidaudio/blob/main/Sources/FluidAudioCLI/FluidAudioCLI.swift): `Sources/FluidAudioCLI/FluidAudioCLI.swift`

### Other components

- **Application**
- **Core ML / ANE**

## Connections

- Application → Audio sources: provides audio
- Application → ASR managers: requests transcription
- Application → Voice activity: requests detection
- Application → Online diarization: requests diarization
- Application → Offline diarization: requests batch analysis
- Application → Speaker embeddings: requests embeddings
- Application → Kokoro TTS: provides text
- Application → Pocket TTS: provides text
- Application → TTS managers: requests synthesis
- Audio sources → ASR managers: supplies audio
- Audio sources → Voice activity: supplies audio
- Audio sources → Online diarization: supplies audio
- Audio sources → Offline diarization: supplies audio
- Model registry → Model downloads: obtains assets
- Model downloads → Model registry: returns assets
- ASR managers → TDT pipeline: runs windows
- TDT pipeline → Core ML / ANE: runs inference
- Streaming ASR → Core ML / ANE: runs inference
- Cohere ASR → Core ML / ANE: runs inference
- Paraformer ASR → Core ML / ANE: runs inference
- Voice activity → Core ML / ANE: runs inference
- Online diarization → Speaker tracking: tracks speakers
- Online diarization → Core ML / ANE: runs inference
- Offline diarization → Core ML / ANE: runs inference
- Speaker embeddings → Core ML / ANE: runs inference
- Kokoro TTS → Core ML / ANE: runs inference
- Pocket TTS → Synthesis state: uses state
- Synthesis state → Core ML / ANE: runs inference
- TTS managers → Core ML / ANE: runs inference
- FluidAudio CLI → ASR managers: dispatches commands
- FluidAudio CLI → Voice activity: dispatches commands
- FluidAudio CLI → Offline diarization: dispatches commands
- FluidAudio CLI → TTS managers: dispatches commands