# huiresearch/flashtts architecture

> Architecture diagram of the GitHub repository huiresearch/flashtts, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-05-12). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/huiresearch/flashtts
- Repository: https://github.com/huiresearch/flashtts

## Overview

##### Project Type and Purpose
- FlashTTS is a Python-based full-stack speech synthesis system. It provides:
- A CLI (`flashtts infer`, `flashtts serve`)
- A REST/Web-UI server (FastAPI + HTML template)
- Multi-backend inference engines (SparkTTS, OrpheusTTS, MegaTTS3)
- Zero-shot voice cloning via “role” data (speaker embeddings + reference audio)
- Core use cases: high-quality TTS, zero-shot speaker cloning, streaming TTS, multi-role dialogue.

##### File Structure Analysis
- Top-level:
- data/roles & data/mega-roles → speaker reference data (audio + features)
- docs/ → user/developer documentation (zh/en)
- examples/ → quick scripts for CLI inference, speed tests
- flashtts/ → main codebase
- setup.py, requirements.txt → packaging/config
- flashtts/ submodules:
- commands/ → CLI entry points & argument parsing
- server/ → FastAPI app, routers, templates
- engine/ → engine selection & orchestration (base + spark/orpheus/mega)
- audio/ → tokenizer/detokenizer, batch processing, I/O utilities
- llm/ → adapters for various LLM backends (vllm, llama-cpp, torch, etc.)
- modules/ → model definitions (encoder, decoder, vocoder, fsq, speaker, VQ)
- logger.py, import_utils.py → logging and dynamic model loading

##### README Insights
- Emphasizes:
- Multi-backend acceleration (VLLM, sglang, llama-cpp, mlx-lm, TRT-LLM)
- High concurrency via dynamic batching, async queues
- Streaming and long-text synthesis support
- FastAPI-based one-command deployment

##### Main Components & Relationships
- External Clients:
- CLI client (shell)
- Web client (browser) → HTTP requests to FastAPI
- Entry Layer:
- flashtts.commands.flashtts_cli → parses user args
- flashtts.server.base_router → defines HTTP endpoints (/infer, /stream)
- Orchestration Layer:
- flashtts.commands.infer & serve → instantiate EngineManager
- EngineManager (flashtts.engine.auto_engine) → selects appropriate `BaseEngine` subclass
- Engine Layer:
- BaseEngine → common API (`synthesize(text, role, params)`)
- SparkEngine, OrpheusEngine, MegaEngine → implement model-specific pipelines
- Internally call:
– flashtts.llm.\* → optional LLM for duration/text planning
– flashtts.audio.tokenizer/detokenizer → text ↔ token IDs ↔ waveform
– flashtts.modules.encoder_decoder → acoustic feature encoder & decoder
– flashtts.modules.blocks/vocos or modules/vq → neural vocoder
- Data & Configuration:
- data/roles & data/mega-roles → lookup speaker embedding or feature
- CLI/server flags → control batch size, devices, dtype, max_length
- Output Layer:
- flashtts.audio.batch_processor & audio_writer → stitch audio frames, write .wav
- HTTP streaming response or file output

##### Architectural Patterns & Principles
- Plugin Pattern: multiple engines and LLM backends via BaseEngine/BaseLLM interfaces
- Pipeline Pattern: Tokenization → Acoustic Model → Vocoder → Audio Writer
- Adapter Pattern: flashtts.llm wraps diverse LLM runtimes under a common API
- Asynchronous & Batch Processing: dynamic batching and async queue in server

## Diagram (Mermaid)

```mermaid
flowchart TD
    %% I/O Layer
    subgraph "I/O Layer"
        CLI["CLI Client"]:::io
        Browser["Web Client (Browser)"]:::io
    end

    %% API Layer
    subgraph "API Layer"
        CLI_CLI["flashtts_cli"]:::control
        InferCmd["infer"]:::control
        ServeCmd["serve"]:::control
        APIRouter["base_router"]:::control
    end

    %% Orchestration Layer
    subgraph "Orchestration"
        EngineMgr["EngineManager\n(auto_engine)"]:::control
    end

    %% Engine Plugins
    subgraph "Engine Plugins"
        Spark["SparkEngine"]:::control
        Orpheus["OrpheusEngine"]:::control
        Mega["MegaEngine"]:::control
    end

    %% LLM Adapter Layer
    subgraph "LLM Adapter Layer"
        BaseLLM["BaseLLM"]:::llm
        VLLM["vllm_generator"]:::llm
        SGLANG["sglang_generator"]:::llm
        LLAMA["llama_cpp_generator"]:::llm
        TORCH["torch_generator"]:::llm
        MLX["mlx_lm_generator"]:::llm
        TRT["trtllm_generator"]:::llm
    end

    %% Model Pipeline
    subgraph "Model Pipeline"
        Tokenizer["Tokenizer"]:::model
        Encoder["FeatEncoder"]:::model
        Decoder["FeatDecoder\n& WaveGenerator"]:::model
        Vocoder["Vocoder Modules"]:::model
    end

    %% Audio I/O
    subgraph "Audio I/O"
        BatchProc["BatchProcessor"]:::io
        AudioWriter["AudioWriter"]:::io
    end

    %% Data Store
    DataStore["Speaker Data\n(data/roles, data/mega-roles)"]:::data

    %% Connections
    CLI --> CLI_CLI
    CLI --> InferCmd
    CLI --> ServeCmd
    Browser --> APIRouter

    CLI_CLI --> EngineMgr
    InferCmd --> EngineMgr
    ServeCmd --> EngineMgr
    APIRouter --> EngineMgr

    EngineMgr --> Spark
    EngineMgr --> Orpheus
    EngineMgr --> Mega

    Spark --> BaseLLM
    Orpheus --> BaseLLM
    Mega --> BaseLLM

    BaseLLM --> VLLM
    BaseLLM --> SGLANG
    BaseLLM --> LLAMA
    BaseLLM --> TORCH
    BaseLLM --> MLX
    BaseLLM --> TRT

    Spark --> Tokenizer
    Orpheus --> Tokenizer
    Mega --> Tokenizer

    Tokenizer --> Encoder
    Encoder --> Decoder
    Decoder --> Vocoder
    Vocoder --> BatchProc
    BatchProc --> AudioWriter

    AudioWriter --> CLI
    AudioWriter --> Browser

    DataStore --> Spark
    DataStore --> Mega

    %% Click Events
    click CLI_CLI "https://github.com/huiresearch/flashtts/blob/master/flashtts.commands/flashtts_cli.py"
    click InferCmd "https://github.com/huiresearch/flashtts/blob/master/flashtts.commands/infer.py"
    click ServeCmd "https://github.com/huiresearch/flashtts/blob/master/flashtts.commands/serve.py"
    click APIRouter "https://github.com/huiresearch/flashtts/blob/master/flashtts.server/base_router.py"
    click EngineMgr "https://github.com/huiresearch/flashtts/blob/master/flashtts.engine/auto_engine.py"
    click BaseLLM "https://github.com/huiresearch/flashtts/blob/master/flashtts.llm/base_llm.py"
    click VLLM "https://github.com/huiresearch/flashtts/blob/master/flashtts.llm/vllm_generator.py"
    click SGLANG "https://github.com/huiresearch/flashtts/blob/master/flashtts.llm/sglang_generator.py"
    click LLAMA "https://github.com/huiresearch/flashtts/blob/master/flashtts.llm/llama_cpp_generator.py"
    click TORCH "https://github.com/huiresearch/flashtts/blob/master/flashtts.llm/torch_generator.py"
    click MLX "https://github.com/huiresearch/flashtts/blob/master/flashtts.llm/mlx_lm_generator.py"
    click TRT "https://github.com/huiresearch/flashtts/blob/master/flashtts.llm/trtllm_generator.py"
    click Tokenizer "https://github.com/huiresearch/flashtts/blob/master/flashtts.audio/tokenizer/spark_tokenizer.py"
    click AudioWriter "https://github.com/huiresearch/flashtts/blob/master/flashtts.server/utils/audio_writer.py"
    click DataStore "https://github.com/huiresearch/flashtts/tree/master/data/roles/"

    %% Styles
    classDef io fill:#ccffcc,stroke:#333,stroke-width:1px;
    classDef control fill:#cce5ff,stroke:#333,stroke-width:1px;
    classDef model fill:#ffdb99,stroke:#333,stroke-width:1px;
    classDef llm fill:#e2ccff,stroke:#333,stroke-width:1px;
    classDef data fill:#b3ffb3,stroke:#333,stroke-width:1px;
```
