How pengchengguo/espnet works
ESPnet is a speech-processing toolkit whose user-facing result is a trained model or an inference output: recognized text, synthesized speech, translated text, enhanced/separated audio, or converted voice. Users prepare recipe data, train a task model, then decode or synthesize. The repository supports both ESPnet1 and ESPnet2; the graph emphasizes ESPnet2’s task-based path and preserves ESPnet1’s major task branches. Relationships involving unsampled subsystems are included only where the README or SOURCE INDEX supports them.
Main entry points: asr_inference.py ASR decoding · tts_inference.py TTS decoding · enh_inference.py Enhancement decoding · asr_train.py ESPnet2 training
Components 22
User workflows
- Task outputs
- Input audio and text
Data and training
- Recipe data prep
egs - Kaldi-style data
- Task configuration
espnet2/tasks - ESPnet2 training
espnet2/bin/asr_train.py - Training engine
espnet2/train/trainer.py - Task model base
espnet2/train/abs_espnet_model.py
Speech tasks
- Speech recognition
espnet2/asr/espnet_model.py - Enhancement and separation
espnet2/enh/espnet_model.py - Speech synthesis
espnet2/tts/transformer.py - Voice conversion
espnet/vc/pytorch_backend/vc.py
Text tasks
- Speech translation
espnet/st/pytorch_backend/st.py - Machine translation
espnet/mt/pytorch_backend/mt.py - Language modeling
espnet2/lm/espnet_model.py
Shared inference
- ASR decoding
espnet2/bin/asr_inference.py - TTS decoding
espnet2/bin/tts_inference.py - Enhancement decoding
espnet2/bin/enh_inference.py - Text tokenization
espnet2/text/build_tokenizer.py - Beam search
espnet/nets/beam_search.py - Feature and audio I/O
espnet/utils/io_utils.py
Other components
- Researcher
Connections 24
- Researcher → Recipe data prep: prepares data
- Recipe data prep → Kaldi-style data: produces data
- Researcher → Task configuration: chooses task
- Kaldi-style data → ESPnet2 training: supplies examples
- Task configuration → ESPnet2 training: configures training
espnet2/tasks/abs_task.py - ESPnet2 training → Training engine: runs training
- Training engine → Task model base: calls model
espnet2/train/trainer.py - Task model base → Speech recognition: task implementation
espnet2/asr/espnet_model.py - Researcher → Input audio and text: provides input
- Input audio and text → ASR decoding: decodes speech
espnet2/bin/asr_inference.py - ASR decoding → Beam search: uses search
espnet2/bin/asr_inference.py - ASR decoding → Text tokenization: converts tokens
espnet2/bin/asr_inference.py - ASR decoding → Task outputs: writes recognition
espnet2/bin/asr_inference.py - Input audio and text → TTS decoding: provides text
- TTS decoding → Speech synthesis: builds TTS model
espnet2/bin/tts_inference.py - TTS decoding → Task outputs: returns speech
espnet2/bin/tts_inference.py - Input audio and text → Enhancement decoding: provides audio
espnet2/bin/enh_inference.py - Enhancement decoding → Enhancement and separation: loads task model
espnet2/bin/enh_inference.py - Enhancement decoding → Task outputs: writes enhanced audio
- Speech translation → Task outputs: produces translation
- Machine translation → Task outputs: produces translation
- Voice conversion → Task outputs: produces converted voice
- Language modeling → ASR decoding: scores hypotheses
- Feature and audio I/O → Input audio and text: loads speech data
espnet/utils/io_utils.py