# netease-youdao/emotivoice architecture

> Architecture diagram of the GitHub repository netease-youdao/emotivoice, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-10-01). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/netease-youdao/emotivoice
- Repository: https://github.com/netease-youdao/emotivoice

## Overview

EmotiVoice is a multilingual, multi-voice text-to-speech engine with prompt-controlled emotion and style. Its main user path is text entry, phoneme conversion, prompt/speaker-conditioned synthesis, and audio output; users can access it through a web demo, batch scripts, or an OpenAI-compatible API. The repository also supports voice-cloning data preparation and model training. Arrows follow explicit README statements or source-index references; unsampled interface and training wiring is left unverified where necessary.

Main entry points: [`inference_am_vocoder_joint.py`](https://github.com/netease-youdao/emotivoice/blob/main/inference_am_vocoder_joint.py) (Batch inference), [`frontend.py`](https://github.com/netease-youdao/emotivoice/blob/main/frontend.py) (Text frontend), [`models/prompt_tts_modified/model_open_source.py`](https://github.com/netease-youdao/emotivoice/blob/main/models/prompt_tts_modified/model_open_source.py) (Prompt TTS model), [`models/prompt_tts_modified/jets.py`](https://github.com/netease-youdao/emotivoice/blob/main/models/prompt_tts_modified/jets.py) (JETS generator)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_interfaces["User interfaces"]
  node_demo["Web demo<br/>[demo_page.py]"]
  node_api["OpenAI API<br/>[openaiapi.py]"]
  node_batch["Batch inference"]
  node_http_demo["HTTP API demo<br/>[TtsDemo.py]"]
end

subgraph group_text["Text processing"]
  node_frontend["Text frontend<br/>[frontend.py]"]
  node_frontend_cn["Chinese frontend<br/>[frontend_cn.py]"]
  node_frontend_en["English frontend<br/>[frontend_en.py]"]
  node_number_conversion["Number conversion<br/>[an2cn.py]"]
  node_text_cleaning["Text cleaners<br/>[cleaners.py]"]
end

subgraph group_synthesis["Speech synthesis"]
  node_config["Synthesis config<br/>[config.py]"]
  node_style_encoder["Style encoder<br/>[simbert.py]"]
  node_tts_generator["JETS generator<br/>[jets.py]"]
  node_tts_model["Prompt TTS model"]
  node_encoder["Text encoder<br/>[encoder.py]"]
  node_alignment["Alignment module<br/>[alignment.py]"]
  node_variance["Variance module<br/>[variance.py]"]
  node_vocoder["HiFi-GAN vocoder<br/>[get_vocoder.py]"]
  node_audio["Generated audio"]
end

subgraph group_data["Data and training"]
  node_data_recipes["Voice data recipes"]
  node_mfa["Alignment preparation"]
  node_features["Acoustic features<br/>[feats.py]"]
  node_training["Model training"]
end

node_user(("TTS user"))

node_user -->|"enters text"| node_demo
node_user -->|"submits batch"| node_batch
node_user -->|"sends request"| node_api
node_user -.->|"uses service demo"| node_http_demo
node_demo -.->|"converts text"| node_frontend
node_batch -->|"prepares phonemes"| node_frontend
node_frontend -->|"dispatches Chinese"| node_frontend_cn
node_frontend -->|"dispatches English"| node_frontend_en
node_frontend_cn -->|"converts numerals"| node_number_conversion
node_frontend_en -.->|"cleans text"| node_text_cleaning
node_batch -->|"encodes prompts"| node_style_encoder
node_batch -->|"synthesizes speech"| node_tts_generator
node_batch -->|"loads settings"| node_config
node_tts_generator -->|"uses model"| node_tts_model
node_tts_generator -->|"uses generator"| node_vocoder
node_tts_model -->|"uses encoder"| node_encoder
node_tts_model -->|"uses alignment"| node_alignment
node_tts_model -->|"uses variance"| node_variance
node_batch -->|"writes audio"| node_audio
node_data_recipes -->|"generates phonemes"| node_frontend
node_data_recipes -.->|"prepares alignment"| node_mfa
node_mfa -.->|"prepares training data"| node_features
node_features -.->|"supports training"| node_training
node_api -.->|"invokes synthesis"| node_batch

click node_demo "https://github.com/netease-youdao/emotivoice/blob/main/demo_page.py"
click node_api "https://github.com/netease-youdao/emotivoice/blob/main/openaiapi.py"
click node_batch "https://github.com/netease-youdao/emotivoice/blob/main/inference_am_vocoder_joint.py"
click node_http_demo "https://github.com/netease-youdao/emotivoice/blob/main/HTTP_API_TtsDemo/apidemo/TtsDemo.py"
click node_frontend "https://github.com/netease-youdao/emotivoice/blob/main/frontend.py"
click node_frontend_cn "https://github.com/netease-youdao/emotivoice/blob/main/frontend_cn.py"
click node_frontend_en "https://github.com/netease-youdao/emotivoice/blob/main/frontend_en.py"
click node_number_conversion "https://github.com/netease-youdao/emotivoice/blob/main/cn2an/an2cn.py"
click node_text_cleaning "https://github.com/netease-youdao/emotivoice/blob/main/text/cleaners.py"
click node_config "https://github.com/netease-youdao/emotivoice/blob/main/config/joint/config.py"
click node_style_encoder "https://github.com/netease-youdao/emotivoice/blob/main/models/prompt_tts_modified/simbert.py"
click node_tts_generator "https://github.com/netease-youdao/emotivoice/blob/main/models/prompt_tts_modified/jets.py"
click node_tts_model "https://github.com/netease-youdao/emotivoice/blob/main/models/prompt_tts_modified/model_open_source.py"
click node_encoder "https://github.com/netease-youdao/emotivoice/blob/main/models/prompt_tts_modified/modules/encoder.py"
click node_alignment "https://github.com/netease-youdao/emotivoice/blob/main/models/prompt_tts_modified/modules/alignment.py"
click node_variance "https://github.com/netease-youdao/emotivoice/blob/main/models/prompt_tts_modified/modules/variance.py"
click node_vocoder "https://github.com/netease-youdao/emotivoice/blob/main/models/hifigan/get_vocoder.py"
click node_data_recipes "https://github.com/netease-youdao/emotivoice/tree/main/data"
click node_mfa "https://github.com/netease-youdao/emotivoice/tree/main/mfa"
click node_features "https://github.com/netease-youdao/emotivoice/blob/main/models/prompt_tts_modified/feats.py"
click node_training "https://github.com/netease-youdao/emotivoice/blob/main/train_am_vocoder_joint.py"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_demo,node_api,node_batch,node_http_demo,node_user toneBlue
class node_frontend,node_frontend_cn,node_frontend_en,node_number_conversion,node_text_cleaning toneAmber
class node_config,node_style_encoder,node_tts_generator,node_tts_model,node_encoder,node_alignment,node_variance,node_vocoder,node_audio toneMint
class node_data_recipes,node_mfa,node_features,node_training toneRose
```

## Components

### User interfaces

- [Web demo](https://github.com/netease-youdao/emotivoice/blob/main/demo_page.py): `demo_page.py`
- [OpenAI API](https://github.com/netease-youdao/emotivoice/blob/main/openaiapi.py): `openaiapi.py`
- [Batch inference](https://github.com/netease-youdao/emotivoice/blob/main/inference_am_vocoder_joint.py): `inference_am_vocoder_joint.py`
- [HTTP API demo](https://github.com/netease-youdao/emotivoice/blob/main/HTTP_API_TtsDemo/apidemo/TtsDemo.py): `HTTP_API_TtsDemo/apidemo/TtsDemo.py`

### Text processing

- [Text frontend](https://github.com/netease-youdao/emotivoice/blob/main/frontend.py): `frontend.py`
- [Chinese frontend](https://github.com/netease-youdao/emotivoice/blob/main/frontend_cn.py): `frontend_cn.py`
- [English frontend](https://github.com/netease-youdao/emotivoice/blob/main/frontend_en.py): `frontend_en.py`
- [Number conversion](https://github.com/netease-youdao/emotivoice/blob/main/cn2an/an2cn.py): `cn2an/an2cn.py`
- [Text cleaners](https://github.com/netease-youdao/emotivoice/blob/main/text/cleaners.py): `text/cleaners.py`

### Speech synthesis

- [Synthesis config](https://github.com/netease-youdao/emotivoice/blob/main/config/joint/config.py): `config/joint/config.py`
- [Style encoder](https://github.com/netease-youdao/emotivoice/blob/main/models/prompt_tts_modified/simbert.py): `models/prompt_tts_modified/simbert.py`
- [JETS generator](https://github.com/netease-youdao/emotivoice/blob/main/models/prompt_tts_modified/jets.py): `models/prompt_tts_modified/jets.py`
- [Prompt TTS model](https://github.com/netease-youdao/emotivoice/blob/main/models/prompt_tts_modified/model_open_source.py): `models/prompt_tts_modified/model_open_source.py`
- [Text encoder](https://github.com/netease-youdao/emotivoice/blob/main/models/prompt_tts_modified/modules/encoder.py): `models/prompt_tts_modified/modules/encoder.py`
- [Alignment module](https://github.com/netease-youdao/emotivoice/blob/main/models/prompt_tts_modified/modules/alignment.py): `models/prompt_tts_modified/modules/alignment.py`
- [Variance module](https://github.com/netease-youdao/emotivoice/blob/main/models/prompt_tts_modified/modules/variance.py): `models/prompt_tts_modified/modules/variance.py`
- [HiFi-GAN vocoder](https://github.com/netease-youdao/emotivoice/blob/main/models/hifigan/get_vocoder.py): `models/hifigan/get_vocoder.py`
- **Generated audio**

### Data and training

- [Voice data recipes](https://github.com/netease-youdao/emotivoice/tree/main/data): `data`
- [Alignment preparation](https://github.com/netease-youdao/emotivoice/tree/main/mfa): `mfa`
- [Acoustic features](https://github.com/netease-youdao/emotivoice/blob/main/models/prompt_tts_modified/feats.py): `models/prompt_tts_modified/feats.py`
- [Model training](https://github.com/netease-youdao/emotivoice/blob/main/train_am_vocoder_joint.py): `train_am_vocoder_joint.py`

### Other components

- **TTS user**

## Connections

- TTS user → Web demo: enters text (evidence: `README.md`)
- TTS user → Batch inference: submits batch (evidence: `README.md`)
- TTS user → OpenAI API: sends request (evidence: `README.md`)
- TTS user → HTTP API demo: uses service demo
- Web demo → Text frontend: converts text
- Batch inference → Text frontend: prepares phonemes (evidence: `README.md`)
- Text frontend → Chinese frontend: dispatches Chinese (evidence: `frontend.py`)
- Text frontend → English frontend: dispatches English (evidence: `frontend.py`)
- Chinese frontend → Number conversion: converts numerals (evidence: `frontend_cn.py`)
- English frontend → Text cleaners: cleans text
- Batch inference → Style encoder: encodes prompts (evidence: `inference_am_vocoder_joint.py`)
- Batch inference → JETS generator: synthesizes speech (evidence: `inference_am_vocoder_joint.py`)
- Batch inference → Synthesis config: loads settings (evidence: `inference_am_vocoder_joint.py`)
- JETS generator → Prompt TTS model: uses model (evidence: `models/prompt_tts_modified/jets.py`)
- JETS generator → HiFi-GAN vocoder: uses generator (evidence: `models/prompt_tts_modified/jets.py`)
- Prompt TTS model → Text encoder: uses encoder (evidence: `models/prompt_tts_modified/model_open_source.py`)
- Prompt TTS model → Alignment module: uses alignment (evidence: `models/prompt_tts_modified/model_open_source.py`)
- Prompt TTS model → Variance module: uses variance (evidence: `models/prompt_tts_modified/model_open_source.py`)
- Batch inference → Generated audio: writes audio (evidence: `inference_am_vocoder_joint.py`)
- Voice data recipes → Text frontend: generates phonemes (evidence: `data/DataBaker/src/step2_get_phoneme.py`)
- Voice data recipes → Alignment preparation: prepares alignment
- Alignment preparation → Acoustic features: prepares training data
- Acoustic features → Model training: supports training
- OpenAI API → Batch inference: invokes synthesis
