# picovoice/tts-latency-benchmark architecture

> Architecture diagram of the GitHub repository picovoice/tts-latency-benchmark, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-06-02). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/picovoice/tts-latency-benchmark
- Repository: https://github.com/picovoice/tts-latency-benchmark

## Overview

##### Project Type and Purpose
- This is an open-source Python benchmarking framework for measuring latency in a text-to-speech (TTS) pipeline that includes a large language model (LLM) and multiple external TTS engines.
- It simulates user–assistant interactions by sampling user queries, generating LLM responses via the OpenAI Chat Completion API, and streaming those tokens into various TTS engines. The goal is to measure “Time to First Token” (TTFT), “First Token to Speech” (FTTS), and aggregate “Voice Assistant Response Time” (VART).

##### File Structure Analysis
Top-level scripts:
- benchmark.py – orchestrates a full run: loads data, drives LLM, invokes TTS engines, times responses, writes JSON results to results/data.
- plot_results.py – reads JSON results, generates plots into results/plots.
Core directories:
- data/ – data/_dataset.py loads and samples user queries from Taskmaster2.
- llm/ – llm/_llm.py handles interactions with OpenAI’s chat-completion API and token streaming.
- tts/ – tts/_tts.py defines a plugin pattern for each TTS engine; tts/_timer.py wraps timing logic.
- audio/ – audio/_audio.py contains helpers for audio buffering or playback (if needed).
Support files: requirements.txt, README.md, .gitignore, LICENSE.

##### Additional Insights from README
- The pipeline has clearly defined stages: Data Ingestion → LLM Response → TTS Synthesis → Metrics Collection → Result Persistence → Plotting.
- External dependencies are all REST or WebSocket services (OpenAI, Amazon Polly, Azure TTS, ElevenLabs, Picovoice Orca).
- The framework is extensible: new TTS engines can be added by implementing the _tts interface.

a. Main Components
1. Data Source & Loader (data/_dataset.py)
2. Benchmark Controller (benchmark.py)
3. LLM Client (llm/_llm.py)
4. TTS Engine Plugins (tts/_tts.py + engine-specific subclasses)
5. Timer/Metrics Collector (tts/_timer.py)
6. Results Store (results/data/*.json)
7. Plotter (plot_results.py)
8. Audio Utilities (audio/_audio.py) – optional for playback
9. External Services (OpenAI, AWS Polly, Azure, ElevenLabs, Picovoice Orca)
b. Relationships & Interactions
- The Benchmark Controller invokes the Data Loader to fetch a batch of user queries.
- For each query:
– Send to LLM Client → receive streaming tokens → record TTFT.
– Pass tokens to TTS Plugin → receive streaming audio → record FTTS.
– Timer aggregates VART = TTFT + FTTS.
- The controller writes per-engine, per-query latencies to JSON.
- The Plotter reads JSON and produces comparison plots.
c. Architectural Patterns & Principles
- Pipeline pattern: sequential stages connected via streamed data.
- Plugin/strategy pattern for TTS engines: a common interface abstracts over multiple external APIs (REST vs. WebSocket).
- Single responsibility: each module handles one stage (data, LLM, TTS, timing, plotting).
- Extensibility: new TTS engines conform to the tts interface.
d. Key Technologies & Libraries
- Python 3.10
- OpenAI Python SDK for chat completion
- boto3 (for Amazon Polly)
- azure-cognitiveservices-speech
- Requests / websockets for ElevenLabs
- Picovoice Orca SDK
- matplotlib or seaborn for plotting

##### Essential Details to Capture
- The two timing metrics (TTFT, FTTS) and their aggregation.
- Streaming nature: token streaming into TTS streaming.
- Plugin architecture for adding new engines.
- Result persistence and plotting as separate stages.
- Configuration inputs (API keys, engine flags) passed into benchmark.py.

## Diagram (Mermaid)

```mermaid
flowchart TB
    %% Local Modules
    DataLoader["Data Loader"]:::local
    BenchmarkController["Benchmark Controller"]:::local
    LLMClient["LLM Client"]:::local
    AudioUtils["Audio Utilities"]:::local
    Plotter["Plotter"]:::local
    RequirementsTxt["requirements.txt"]:::local
    Readme["README.md"]:::local

    %% Plugin Interfaces
    TTSInterface["TTS Engine Interface"]:::plugin
    Timer["Timer/Metrics Collector"]:::plugin

    %% Data Store
    ResultsStore["Results Store (JSON)"]:::store

    %% External Services
    OpenAI["OpenAI Chat API"]:::external
    AWSPolly["AWS Polly"]:::external
    AzureTTS["Azure TTS"]:::external
    ElevenLabs["ElevenLabs"]:::external
    Picovoice["Picovoice Orca"]:::external

    %% Flow
    BenchmarkController -->|"fetch batch of queries"| DataLoader
    DataLoader -->|"user queries"| BenchmarkController

    BenchmarkController -->|"send user query"| LLMClient
    LLMClient -->|"TTFT metric"| Timer
    LLMClient -->|"streaming tokens"| TTSInterface
    TTSInterface -->|"FTTS metric"| Timer
    TTSInterface -->|"audio bytes"| AudioUtils
    Timer -->|"metrics"| BenchmarkController

    BenchmarkController -->|"write latencies"| ResultsStore
    Plotter -->|"read JSON"| ResultsStore
    ResultsStore -->|"JSON data"| Plotter

    BenchmarkController -->|"trigger plotting"| Plotter

    LLMClient -->|"chat completion"| OpenAI
    TTSInterface -->|"synthesize"| AWSPolly
    TTSInterface -->|"synthesize"| AzureTTS
    TTSInterface -->|"synthesize"| ElevenLabs
    TTSInterface -->|"synthesize"| Picovoice

    %% Click Events
    click DataLoader "https://github.com/picovoice/tts-latency-benchmark/blob/main/data/_dataset.py"
    click BenchmarkController "https://github.com/picovoice/tts-latency-benchmark/blob/main/benchmark.py"
    click LLMClient "https://github.com/picovoice/tts-latency-benchmark/blob/main/llm/_llm.py"
    click TTSInterface "https://github.com/picovoice/tts-latency-benchmark/blob/main/tts/_tts.py"
    click Timer "https://github.com/picovoice/tts-latency-benchmark/blob/main/tts/_timer.py"
    click ResultsStore "https://github.com/picovoice/tts-latency-benchmark/blob/main/results/data/*.json"
    click Plotter "https://github.com/picovoice/tts-latency-benchmark/blob/main/plot_results.py"
    click AudioUtils "https://github.com/picovoice/tts-latency-benchmark/blob/main/audio/_audio.py"
    click RequirementsTxt "https://github.com/picovoice/tts-latency-benchmark/blob/main/requirements.txt"
    click Readme "https://github.com/picovoice/tts-latency-benchmark/blob/main/README.md"

    %% Styles
    classDef local fill:#ADD8E6,stroke:#333,stroke-width:1px
    classDef plugin fill:#90EE90,stroke:#333,stroke-width:1px
    classDef store fill:#FFA500,stroke:#333,stroke-width:1px
    classDef external fill:#87CEEB,stroke:#333,stroke-width:1px,stroke-dasharray: 5 5
```
