How espressif/esp-sr works
Project Type & Purpose
• This is an ESP-IDF component library (“esp-sr”) providing on-device AI speech functionality (Audio Front-End, Wake-Word, VAD, Speech-Command Recognition) plus a separate TTS component (“esp-tts”).
• It targets multiple Espressif chips (ESP32, ESP32-S2/S3/P4, ESP32-C3/C5/C6) and is packaged as a reusable component with prebuilt libraries and model assets.
High-Level Component Breakdown
A. Hardware/OS Layer
• Underneath, “ESP-IDF RTOS” (FreeRTOS + IDF drivers).
B. Build & Packaging Layer
• “esp-sr Component”
– CMakeLists.txt & component.mk (ESP-IDF build scripts)
– Kconfig.projbuild (config options)
1. Core Sources (src/, include/)
2. Prebuilt Algorithm Libraries (lib/esp32*, lib/esp32c3*, …)
C. Model Assets Layer
• Inside, FST files, MultiNet models, VADNet models, WakeNet models, TTS voice data.
D. Tooling & Utilities Layer
E. Documentation Layer
F. Application Layer
• “Example Applications” (test_apps/esp-sr, test_apps/esp-tts, test_apps/esp32c5)
Data Flow & Interactions
1. Audio Input (Mic) → Audio Front-End (AFE)
2. AFE → Branch
• → WakeNet (wake-word detection)
• → VADNet (voice-activity detection) → MultiNet (speech-command recognition)
3. Command Output → User Application
4. Text Input (User App) → esp-tts Component (TTS Parser → TTS Player → Audio Output)
Architectural Patterns & Design Principles
• Modular component design: each algorithm is a standalone library with C API in include/*.
• Build-time configuration via CMake + Kconfig, producing per-target static libs.
• Model-data separation: algorithm binaries + external model assets.
• Test harnesses: Python/pytest + C/C++ tests to validate each algorithm.
Essential Elements
• Build scripts & config (CMakeLists.txt, component.mk, Kconfig).
• Core API layers (include/, src/).
• Precompiled algorithm libraries segmented by chip targets (lib/esp32*, lib/esp32c3*, …).
• Model storage & preparation tools (model/, tool/).
• Runtime flow: Audio Input → AFE → WakeNet/VADNet/MultiNet → Output.
• TTS flow: Text → Parser → Player → Audio Output.
• CI/CD pipelines (.github, gitlab-ci.yml) and test runners (pytest.ini, test_apps/).
• Documentation subsystem (docs/, Sphinx).