# erax-ai/vif5tts architecture

> Architecture diagram of the GitHub repository erax-ai/vif5tts, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-05-18). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/erax-ai/vif5tts
- Repository: https://github.com/erax-ai/vif5tts

## Overview

##### Project type and purpose
- This is a machine-learning text-to-speech (TTS) toolkit built around the F5-TTS architecture, extended for Vietnamese (“EraX-Smile-Female-F5-V1.0”).
- It provides modules for training (dataset preparation, trainer, pruning), evaluation, inference (CLI, Gradio, wrapper), and real-time streaming (FastAPI, socket).

##### File structure
- Top-level directories:
- `src/f5_tts/model` – core model code (backbones, modules, trainer).
- `src/f5_tts/train` – training pipelines & dataset pre-processing scripts.
- `src/f5_tts/infer` – inference wrappers (CLI, Gradio), example configs and audio.
- `src/f5_tts/eval` – evaluation scripts for benchmarks.
- `src/streaming` – real-time streaming server (FastAPI) and client (JS/HTML).
- `src/model_pruning` – utilities to prune and convert checkpoints.
- `src/socket_server`, `src/socket_client` – alternative streaming via sockets.
- Root config files: `pyproject.toml`, `ruff.toml`, and YAML config templates.

##### README for insights
- Describes how to run FastAPI streaming service.
- Shows Python example for CLI-based inference using `F5TTSWrapper` and `Vinorm` text normalizer.
- Highlights dependencies: PyTorch, fastapi/uvicorn, piper-tts, langchain, underthesea, vinorm.

## Diagram (Mermaid)

```mermaid
flowchart TB
    %% Data Layer
    subgraph "Data Layer"
        direction TB
        RawAudioText(("Raw Audio + Text")):::data
        PrepLJS["prepare_ljspeech.py"]:::dataTool
        PrepLibri["prepare_libritts.py"]:::dataTool
        PrepWenet["prepare_wenetspeech4tts.py"]:::dataTool
        PrepEmilia["prepare_emilia.py"]:::dataTool
        PrepCSV["prepare_csv_wavs.py"]:::dataTool
    end

    %% Training Pipeline
    subgraph "Training Pipeline"
        direction TB
        DatasetLoader["dataset.py"]:::train
        Alignment["alignment_utils.py"]:::train
        Duration["duration_predictor.py"]:::train
        Trainer["trainer.py"]:::train
        CheckpointStore[(Checkpoints)]:::store
    end

    %% Pruning & Conversion
    subgraph "Model Pruning & Conversion"
        direction TB
        Pruner["excellent_definitive-f5tts-pruner.py"]:::util
        ResetEpoch["reset_f5tts_epoch_enhanced.py"]:::util
        InspectPT["inspect_pt.py"]:::util
        ToSafe["pt_to_safetensor.py"]:::util
        PrunedCheckpoints[(Pruned Checkpoints)]:::store
    end

    %% Evaluation Suite
    subgraph "Evaluation Suite"
        direction TB
        EvalBatch["eval_infer_batch.py"]:::eval
        EvalLibri["eval_librispeech_test_clean.py"]:::eval
        EvalUTMOS["eval_utmos.py"]:::eval
        EvalSeed["eval_seedtts_testset.py"]:::eval
        EvalSpeaker["ecapa_tdnn.py"]:::eval
        EvalUtils["utils_eval.py"]:::eval
        EvalResults[(Metrics Results)]:::store
    end

    %% Inference Layer
    subgraph "Inference Layer"
        direction TB
        Wrapper["f5tts_wrapper.py"]:::inference
        CLI["infer_cli.py"]:::inference
        Gradio["infer_gradio.py"]:::inference
        SpeechEdit["speech_edit.py"]:::inference
        InferUtils["utils_infer.py"]:::inference
        Examples["examples/"]:::inference
        AudioOut(("Audio Output")):::out
    end

    %% Streaming Service
    subgraph "Streaming Service"
        direction TB
        FastAPI["f5tts-fastapi-server.py"]:::stream
        StreamWrapper["f5tts_wrapper.py"]:::stream
        HTTPClient["client.html"]:::stream
        NodeClient["f5tts-node-client.js"]:::stream
        SocketServ["socket_server.py"]:::stream
        SocketClient["socket_client.py"]:::stream
    end

    %% Utilities & Scripts
    subgraph "Utilities & Scripts"
        direction TB
        CountParams["count_params_gflops.py"]:::util
        CountEpoch["count_max_epoch.py"]:::util
        APIExports["api.py"]:::util
    end

    %% External Dependencies
    Vinorm(("Vinorm Service")):::external
    Piper(("Piper-TTS Server")):::external

    %% Data Flow Connections
    RawAudioText --> PrepLJS --> DatasetLoader
    RawAudioText --> PrepLibri --> DatasetLoader
    RawAudioText --> PrepWenet --> DatasetLoader
    RawAudioText --> PrepEmilia --> DatasetLoader
    RawAudioText --> PrepCSV --> DatasetLoader

    DatasetLoader --> Alignment --> Duration --> Trainer --> CheckpointStore
    CheckpointStore --> Pruner --> PrunedCheckpoints
    PrunedCheckpoints --> ToSafe --> PrunedCheckpoints
    PrunedCheckpoints --> ResetEpoch --> PrunedCheckpoints
    PrunedCheckpoints --> InspectPT

    CheckpointStore --> Wrapper
    Wrapper --> AudioOut
    Wrapper --> CLI --> AudioOut
    Wrapper --> Gradio --> AudioOut
    Wrapper --> SpeechEdit --> AudioOut
    Wrapper --> InferUtils

    CheckpointStore --> EvalBatch --> EvalResults
    CheckpointStore --> EvalLibri --> EvalResults
    CheckpointStore --> EvalUTMOS --> EvalResults
    CheckpointStore --> EvalSeed --> EvalResults
    CheckpointStore --> EvalSpeaker --> EvalResults

    AudioOut --> FastAPI --> HTTPClient
    AudioOut --> FastAPI --> NodeClient
    FastAPI --> StreamWrapper --> AudioOut
    AudioOut --> SocketServ --> SocketClient

    CLI --> Vinorm
    Gradio --> Vinorm
    SpeechEdit --> Piper

    %% Click Events
    click PrepLJS "https://github.com/erax-ai/vif5tts/blob/main/src/f5_tts/train/datasets/prepare_ljspeech.py"
    click PrepLibri "https://github.com/erax-ai/vif5tts/blob/main/src/f5_tts/train/datasets/prepare_libritts.py"
    click PrepWenet "https://github.com/erax-ai/vif5tts/blob/main/src/f5_tts/train/datasets/prepare_wenetspeech4tts.py"
    click PrepEmilia "https://github.com/erax-ai/vif5tts/blob/main/src/f5_tts/train/datasets/prepare_emilia.py"
    click PrepCSV "https://github.com/erax-ai/vif5tts/blob/main/src/f5_tts/train/datasets/prepare_csv_wavs.py"

    click DatasetLoader "https://github.com/erax-ai/vif5tts/blob/main/src/f5_tts/model/dataset.py"
    click Alignment "https://github.com/erax-ai/vif5tts/blob/main/src/f5_tts/model/alignment_utils.py"
    click Duration "https://github.com/erax-ai/vif5tts/blob/main/src/f5_tts/model/duration_predictor.py"
    click Trainer "https://github.com/erax-ai/vif5tts/blob/main/src/f5_tts/model/trainer.py"
    click ToSafe "https://github.com/erax-ai/vif5tts/blob/main/src/model_pruning/pt_to_safetensor.py"
    click Pruner "https://github.com/erax-ai/vif5tts/blob/main/src/model_pruning/excellent_definitive-f5tts-pruner.py"
    click ResetEpoch "https://github.com/erax-ai/vif5tts/blob/main/src/model_pruning/reset_f5tts_epoch_enhanced.py"
    click InspectPT "https://github.com/erax-ai/vif5tts/blob/main/src/model_pruning/inspect_pt.py"

    click EvalBatch "https://github.com/erax-ai/vif5tts/blob/main/src/f5_tts/eval/eval_infer_batch.py"
    click EvalLibri "https://github.com/erax-ai/vif5tts/blob/main/src/f5_tts/eval/eval_librispeech_test_clean.py"
    click EvalUTMOS "https://github.com/erax-ai/vif5tts/blob/main/src/f5_tts/eval/eval_utmos.py"
    click EvalSeed "https://github.com/erax-ai/vif5tts/blob/main/src/f5_tts/eval/eval_seedtts_testset.py"
    click EvalSpeaker "https://github.com/erax-ai/vif5tts/blob/main/src/f5_tts/eval/ecapa_tdnn.py"
    click EvalUtils "https://github.com/erax-ai/vif5tts/blob/main/src/f5_tts/eval/utils_eval.py"

    click Wrapper "https://github.com/erax-ai/vif5tts/blob/main/src/f5_tts/infer/f5tts_wrapper.py"
    click CLI "https://github.com/erax-ai/vif5tts/blob/main/src/f5_tts/infer/infer_cli.py"
    click Gradio "https://github.com/erax-ai/vif5tts/blob/main/src/f5_tts/infer/infer_gradio.py"
    click SpeechEdit "https://github.com/erax-ai/vif5tts/blob/main/src/f5_tts/infer/speech_edit.py"
    click InferUtils "https://github.com/erax-ai/vif5tts/blob/main/src/f5_tts/infer/utils_infer.py"
    click Examples "https://github.com/erax-ai/vif5tts/tree/main/src/f5_tts/infer/examples/"

    click FastAPI "https://github.com/erax-ai/vif5tts/blob/main/src/streaming/f5tts-fastapi-server.py"
    click HTTPClient "https://github.com/erax-ai/vif5tts/blob/main/src/streaming/client.html"
    click NodeClient "https://github.com/erax-ai/vif5tts/blob/main/src/streaming/f5tts-node-client.js"
    click SocketServ "https://github.com/erax-ai/vif5tts/blob/main/src/f5_tts/socket_server.py"
    click SocketClient "https://github.com/erax-ai/vif5tts/blob/main/src/f5_tts/socket_client.py"

    click CountParams "https://github.com/erax-ai/vif5tts/blob/main/src/f5_tts/scripts/count_params_gflops.py"
    click CountEpoch "https://github.com/erax-ai/vif5tts/blob/main/src/f5_tts/scripts/count_max_epoch.py"
    click APIExports "https://github.com/erax-ai/vif5tts/blob/main/src/f5_tts/api.py"

    %% Styles
    classDef data fill:#BBDEFB,stroke:#1E88E5,color:#000
    classDef dataTool fill:#E3F2FD,stroke:#90CAF9,color:#000
    classDef train fill:#90CAF9,stroke:#42A5F5,color:#000
    classDef store fill:#80CBC4,stroke:#009688,color:#000
    classDef util fill:#CFD8DC,stroke:#78909C,color:#000
    classDef eval fill:#BDBDBD,stroke:#616161,color:#000
    classDef inference fill:#A5D6A7,stroke:#43A047,color:#000
    classDef stream fill:#FFCC80,stroke:#FB8C00,color:#000
    classDef external fill:#F8BBD0,stroke:#D81B60,color:#000
    classDef out fill:#C8E6C9,stroke:#66BB6A,color:#000
```
