# atlonxp/f5-tts architecture

> Architecture diagram of the GitHub repository atlonxp/f5-tts, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-04-29). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/atlonxp/f5-tts
- Repository: https://github.com/atlonxp/f5-tts

## Overview

##### Project type and purpose
- F5-TTS is a full-stack TTS framework. It provides:
– A Python library (`src/f5_tts/…`) for training, inference, evaluation.
– Command-line and Gradio-based front ends.
– A runtime deployment via Triton + TensorRT-LLM.
– A Django-based REST API (in `api/…`) for serving/storing inference results.

##### Main components
a. Clients & Interfaces
- CLI client (`f5-tts_infer-cli`, `socket_client.py`)
- Gradio web UI (`f5-tts_infer-gradio`, `infer_gradio.py`)
- Django Admin/API front end (`api/`)
b. Core Library
- `src/f5_tts/…` – model definitions (CFM, ConvNeXtV2, UNet, DiT), dataset loaders, trainer, inference utilities, scripts.
- Third-party modules (`src/third_party/deep_phonemizer`, `BigVGAN`, `ai4thai`)
c. Model Runtime
- PyTorch inference pipeline (CPU/GPU)
- Triton + TensorRT-LLM server (`src/f5_tts/runtime/triton_trtllm/…`) exposing gRPC/HTTP endpoints
- Socket-server mode (`socket_server.py`)
d. Data & Storage
- SQLite (`api/db.sqlite3`) – metadata, usage logs
- Media storage (`api/media/…`) – uploaded/​generated WAVs
- Static assets (`api/staticfiles/…`) for Django docs & swagger UI
- Model checkpoints (`ckpts/`), dataset files (`data/`), demo files (`demo/`)
e. External Services
- Hugging Face Hub (model hosting)
- GitHub Container Registry (Docker image)
- Pre-commit CI, GitHub Actions

##### Interactions & data flow
1. User issues a TTS request via:
- CLI → directly calls `src/f5_tts/infer/utils_infer.py` or via socket server
- Gradio UI → `infer_gradio.py` → same core library
- HTTP to Django API → Django views (`api/tts/api/tts.py`) → invokes core library → stores result to SQLite & media folder
- gRPC/HTTP to Triton server → Triton loads serialized models (`model_repo_f5_tts`) → returns mel-spectrogram/audio bytes
2. Core library routes calls:
- To PyTorch modules (model/backbones, cfm, modules, vocoder)
- To third-party phonemizer or vocoder (BigVGAN) as needed
3. Training & Evaluation
- Scripts under `src/f5_tts/train/…` call dataset preparers → trainer → save checkpoints
- Evaluation modules under `src/f5_tts/eval/…` load models & datasets → compute metrics

##### Architectural patterns & design principles
- Modular package separation: library vs. interface vs. runtime vs. API
- Adapter pattern for different inference backends (direct PyTorch vs. socket vs. Triton)
- MVC-style for Django API: models (usagelog, speaker, etc.) → views (API endpoints) → templates/static for docs
- Configuration-as-code with YAML (in `configs/`) and TOML (in `infer/examples/`)

- All major components clearly labeled
- On staticfiles for API documentation (Swagger/Redoc)

## Diagram (Mermaid)

```mermaid
flowchart TB
    %% Client Layer
    subgraph "Client Layer"
        direction TB
        CLI["CLI Client"]:::client
        SocketClient["Socket Client"]:::client
        GradioUI["Gradio Web UI"]:::client
    end

    %% API / Interface Layer
    subgraph "API / Interface Layer"
        direction TB
        subgraph "Django REST API"
            direction TB
            DjangoCore["Django Core (settings, URLs)"]:::interface
            API_TTS["TTS Endpoints"]:::interface
            API_Speaker["Speaker Endpoints"]:::interface
            API_Models["ORM Models"]:::interface
            SQLite["SQLite DB"]:::storage
            MediaStore["Media Storage"]:::storage
            StaticFiles["Static Assets"]:::storage
        end
        SocketServer["Socket Server"]:::interface
        TritonServer["Triton + TensorRT-LLM Server"]:::interface
        DjangoGradio["Gradio API Endpoint"]:::interface
    end

    %% Core Library Layer
    subgraph "Core Library Layer"
        direction TB
        CoreModel["Model Modules"]:::core
        InferUtils["Inference Utilities"]:::core
        ThirdParty["Third-Party Adapters"]:::core
    end

    %% Model & Data Layer
    subgraph "Model & Data Layer"
        direction TB
        Checkpoints["Model Checkpoints (ckpts/)"]:::data
        DatasetFiles["Datasets (data/)"]:::data
        DemoFiles["Demo Files (demo/)"]:::data
        ModelRepo["Triton Model Repo"]:::data
    end

    %% External Services
    subgraph "External Services"
        direction TB
        HF["Hugging Face Hub"]:::external
        DockerReg["GitHub Container Registry"]:::external
    end

    %% Data Flows
    CLI -->|"Python call"| InferUtils
    CLI -->|"Socket"| SocketServer
    SocketClient -->|"Socket"| SocketServer
    GradioUI -->|"Python call"| InferUtils
    GradioUI -->|"HTTP"| DjangoGradio
    DjangoGradio -->|"Internal call"| InferUtils
    API_TTS -->|"Calls"| InferUtils
    SocketServer -->|"Calls"| InferUtils
    TritonServer -->|"gRPC/HTTP"| ModelRepo
    TritonServer -->|"Uses"| InferUtils
    InferUtils -->|"Loads models"| CoreModel
    CoreModel -->|"Uses"| ThirdParty
    API_TTS -->|"Stores"| SQLite
    API_TTS -->|"Writes"| MediaStore
    API_TTS -->|"Serves"| StaticFiles
    InferUtils -->|"Reads configs"| DatasetFiles
    InferUtils -->|"Reads checkpoints"| Checkpoints
    CoreModel -->|"Fetches"| HF
    DockerReg -->|"Hosts image"| TritonServer

    %% Click Events
    click CLI "https://github.com/atlonxp/f5-tts/blob/main/src/f5_tts/infer/infer_cli.py"
    click SocketClient "https://github.com/atlonxp/f5-tts/blob/main/src/f5_tts/socket_client.py"
    click GradioUI "https://github.com/atlonxp/f5-tts/blob/main/src/f5_tts/infer/infer_gradio.py"
    click DjangoGradio "https://github.com/atlonxp/f5-tts/blob/main/src/f5_tts/tts/infer_gradio_api.py"
    click DjangoCore "https://github.com/atlonxp/f5-tts/tree/main/api/core/"
    click API_TTS "https://github.com/atlonxp/f5-tts/blob/main/api/tts/api/tts.py"
    click API_Speaker "https://github.com/atlonxp/f5-tts/blob/main/api/tts/api/speaker.py"
    click API_Models "https://github.com/atlonxp/f5-tts/tree/main/api/tts/models/"
    click SQLite "https://github.com/atlonxp/f5-tts/blob/main/api/db.sqlite3"
    click MediaStore "https://github.com/atlonxp/f5-tts/tree/main/api/media/"
    click StaticFiles "https://github.com/atlonxp/f5-tts/tree/main/api/staticfiles/"
    click CoreModel "https://github.com/atlonxp/f5-tts/blob/main/src/f5_tts/model/cfm.py"
    click InferUtils "https://github.com/atlonxp/f5-tts/blob/main/src/f5_tts/infer/utils_infer.py"
    click ThirdParty "https://github.com/atlonxp/f5-tts/tree/main/src/third_party/deep_phonemizer/"
    click ModelRepo "https://github.com/atlonxp/f5-tts/tree/main/src/f5_tts/runtime/triton_trtllm/model_repo_f5_tts/"
    click SocketServer "https://github.com/atlonxp/f5-tts/blob/main/src/f5_tts/socket_server.py"
    click TritonServer "https://github.com/atlonxp/f5-tts/blob/main/src/f5_tts/runtime/triton_trtllm/Dockerfile.server"

    %% Styles
    classDef client fill:#D6E9FE,stroke:#2171B5,color:#000;
    classDef interface fill:#E8F4DA,stroke:#4C9A2A,color:#000;
    classDef core fill:#FFF3BF,stroke:#D39E00,color:#000;
    classDef data fill:#FDE2E2,stroke:#D7263D,color:#000;
    classDef storage fill:#F0F0F0,stroke:#888,color:#000,stroke-dasharray: 2 2;
    classDef external fill:#E2F0CB,stroke:#7AA874,color:#000;
```
