# espressif/esp-sr architecture

> Architecture diagram of the GitHub repository espressif/esp-sr, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-06-20). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/espressif/esp-sr
- Repository: https://github.com/espressif/esp-sr

## Overview

##### Project Type & Purpose
- This is an ESP-IDF component library (“esp-sr”) providing on-device AI speech functionality (Audio Front-End, Wake-Word, VAD, Speech-Command Recognition) plus a separate TTS component (“esp-tts”).
- It targets multiple Espressif chips (ESP32, ESP32-S2/S3/P4, ESP32-C3/C5/C6) and is packaged as a reusable component with prebuilt libraries and model assets.

##### High-Level Component Breakdown

A. Hardware/OS Layer
- Underneath, “ESP-IDF RTOS” (FreeRTOS + IDF drivers).

B. Build & Packaging Layer
- “esp-sr Component”
– CMakeLists.txt & component.mk (ESP-IDF build scripts)
– Kconfig.projbuild (config options)
1. Core Sources (src/, include/)
2. Prebuilt Algorithm Libraries (lib/esp32*, lib/esp32c3*, …)

C. Model Assets Layer
- Inside, FST files, MultiNet models, VADNet models, WakeNet models, TTS voice data.

D. Tooling & Utilities Layer

E. Documentation Layer

F. Application Layer
- “Example Applications” (test_apps/esp-sr, test_apps/esp-tts, test_apps/esp32c5)

##### Data Flow & Interactions
1. Audio Input (Mic) → Audio Front-End (AFE)
2. AFE → Branch
- → WakeNet (wake-word detection)
- → VADNet (voice-activity detection) → MultiNet (speech-command recognition)
3. Command Output → User Application
4. Text Input (User App) → esp-tts Component (TTS Parser → TTS Player → Audio Output)

##### Architectural Patterns & Design Principles
- Modular component design: each algorithm is a standalone library with C API in include/*.
- Build-time configuration via CMake + Kconfig, producing per-target static libs.
- Model-data separation: algorithm binaries + external model assets.
- Test harnesses: Python/pytest + C/C++ tests to validate each algorithm.

##### Essential Elements
- Build scripts & config (CMakeLists.txt, component.mk, Kconfig).
- Core API layers (include/, src/).
- Precompiled algorithm libraries segmented by chip targets (lib/esp32*, lib/esp32c3*, …).
- Model storage & preparation tools (model/, tool/).
- Runtime flow: Audio Input → AFE → WakeNet/VADNet/MultiNet → Output.
- TTS flow: Text → Parser → Player → Audio Output.
- CI/CD pipelines (.github, gitlab-ci.yml) and test runners (pytest.ini, test_apps/).
- Documentation subsystem (docs/, Sphinx).

## Diagram (Mermaid)

```mermaid
flowchart TB
  %% Hardware/OS Layer
  subgraph "Hardware/OS Layer"
    SoCTargets["SoC Targets: ESP32, ESP32-S2, ESP32-S3, ESP32-P4, ESP32-C3, ESP32-C5, ESP32-C6"]:::hw
    ESPIDF["ESP-IDF RTOS<br/>(FreeRTOS + IDF drivers)"]:::hw
  end
  SoCTargets --> ESPIDF

  %% Build & Packaging Layer
  subgraph "Build & Packaging Layer"
    subgraph "esp-sr Component" 
      CMake["CMakeLists.txt"]:::component
      CompMk["component.mk"]:::component
      Kconfig["Kconfig.projbuild"]:::component

      subgraph "Core Sources"
        Src["src/"]:::component
        Inc["include/"]:::component
      end

      subgraph "Prebuilt Algorithm Libraries"
        Lib32["lib/esp32/"]:::component
        LibC3["lib/esp32c3/"]:::component
        LibC5["lib/esp32c5/"]:::component
        LibC6["lib/esp32c6/"]:::component
        LibP4["lib/esp32p4/"]:::component
        LibS2["lib/esp32s2/"]:::component
        LibS3["lib/esp32s3/"]:::component
      end
    end

    subgraph "esp-tts Component"
      TtsCMake["esp-tts/CMakeLists.txt"]:::component
      TtsMk["esp-tts/component.mk"]:::component
      TtsReadme["esp-tts/README.md"]:::component

      subgraph "Chinese TTS Libraries"
        CT32["esp-tts/esp_tts_chinese/esp32/"]:::component
        CTC3["esp-tts/esp_tts_chinese/esp32c3/"]:::component
        CTC5["esp-tts/esp_tts_chinese/esp32c5/"]:::component
        CTC6["esp-tts/esp_tts_chinese/esp32c6/"]:::component
        CTP4["esp-tts/esp_tts_chinese/esp32p4/"]:::component
        CTS2["esp-tts/esp_tts_chinese/esp32s2/"]:::component
        CTS3["esp-tts/esp_tts_chinese/esp32s3/"]:::component
        CTInc["esp-tts/esp_tts_chinese/include/"]:::component
      end
    end
  end

  %% Model Assets Layer
  subgraph "Model Assets Layer"
    ModelRepo["Model Repository"]:::model
    subgraph "Models"
      MN["model/multinet_model/"]:::model
      VN["model/vadnet_model/"]:::model
      WN["model/wakenet_model/"]:::model
      NS["model/nsnet_model/"]:::model
      Target["model/target/"]:::model
    end
    Pack["pack_model.py"]:::model
  end

  %% Tooling & Utilities Layer
  subgraph "Tooling & Utilities Layer"
    subgraph "Model-Preparation Tools"
      PrepFST["tool/prepare_for_fst.py"]:::tools
      G2P["tool/multinet_g2p.py"]:::tools
      Pinyin["tool/multinet_pinyin.py"]:::tools
      subgraph "FST Resources"
        BPE["tool/fst/bpe.model"]:::tools
        Cmds["tool/fst/commands_list.txt"]:::tools
        PrepFST2["tool/fst/prepare_for_fst.py"]:::tools
      end
    end

    subgraph "CI/CD & Testing"
      GH[".github/workflows/"]:::tools
      GL[".gitlab-ci.yml"]:::tools
      Utils["ci/utils.sh"]:::tools
      PytestIni["pytest.ini"]:::tools
      Conftest["conftest.py"]:::tools
    end
  end

  %% Documentation Layer
  subgraph "Documentation Layer"
    Docs["docs/"]:::docs
    Conf["docs/conf_common.py"]:::docs
    subgraph "En & Zh Docs"
      EN["docs/en/"]:::docs
      ZH["docs/zh_CN/"]:::docs
    end
    CheckChars["docs/check_doc_chars.py"]:::docs
    Sync["docs/check_lang_folder_sync.sh"]:::docs
    Req["docs/requirements.txt"]:::docs
    Chart["docs/generate_chart.py"]:::docs
  end

  %% Application Layer
  subgraph "Example Applications"
    AppSR["test_apps/esp-sr/"]:::app
    AppTTS["test_apps/esp-tts/"]:::app
    AppC5["test_apps/esp32c5/"]:::app
  end

  %% Data Flow Runtime
  subgraph "Runtime Data Flows"
    AudioIn["Microphone Input"]:::runtime
    AFE["Audio Front-End (AFE)"]:::runtime
    WakeNet["WakeNet Detector"]:::runtime
    VADNet["VADNet Detector"]:::runtime
    MultiNet["MultiNet Recognizer"]:::runtime
    CmdOut["Command Output → App"]:::runtime
    TextIn["Text Input → App"]:::runtime
    TTSParser["TTS Parser"]:::runtime
    TTSPlayer["TTS Player"]:::runtime
    AudioOut["Audio Output"]:::runtime
  end

  %% Runtime arrows
  AudioIn -->|audio_buffer_t| AFE
  AFE -->|buffer| WakeNet
  AFE -->|buffer| VADNet
  VADNet -->|speech_buffer| MultiNet
  MultiNet -->|command| CmdOut
  TextIn -->|text| TTSParser
  TTSParser -->|parsed_data| TTSPlayer
  TTSPlayer -->|audio| AudioOut

  %% Applications link
  CMake --> AppSR
  TtsCMake --> AppTTS
  CMake --> AppC5

  %% Click Events
  click CMake "https://github.com/espressif/esp-sr/blob/master/CMakeLists.txt"
  click CompMk "https://github.com/espressif/esp-sr/blob/master/component.mk"
  click Kconfig "https://github.com/espressif/esp-sr/blob/master/Kconfig.projbuild"
  click Src "https://github.com/espressif/esp-sr/tree/master/src/"
  click Inc "https://github.com/espressif/esp-sr/tree/master/include/"
  click Lib32 "https://github.com/espressif/esp-sr/tree/master/lib/esp32/"
  click LibC3 "https://github.com/espressif/esp-sr/tree/master/lib/esp32c3/"
  click LibC5 "https://github.com/espressif/esp-sr/tree/master/lib/esp32c5/"
  click LibC6 "https://github.com/espressif/esp-sr/tree/master/lib/esp32c6/"
  click LibP4 "https://github.com/espressif/esp-sr/tree/master/lib/esp32p4/"
  click LibS2 "https://github.com/espressif/esp-sr/tree/master/lib/esp32s2/"
  click LibS3 "https://github.com/espressif/esp-sr/tree/master/lib/esp32s3/"
  click TtsCMake "https://github.com/espressif/esp-sr/blob/master/esp-tts/CMakeLists.txt"
  click TtsMk "https://github.com/espressif/esp-sr/blob/master/esp-tts/component.mk"
  click TtsReadme "https://github.com/espressif/esp-sr/blob/master/esp-tts/README.md"
  click CT32 "https://github.com/espressif/esp-sr/tree/master/esp-tts/esp_tts_chinese/esp32/"
  click CTC3 "https://github.com/espressif/esp-sr/tree/master/esp-tts/esp_tts_chinese/esp32c3/"
  click CTC5 "https://github.com/espressif/esp-sr/tree/master/esp-tts/esp_tts_chinese/esp32c5/"
  click CTC6 "https://github.com/espressif/esp-sr/tree/master/esp-tts/esp_tts_chinese/esp32c6/"
  click CTP4 "https://github.com/espressif/esp-sr/tree/master/esp-tts/esp_tts_chinese/esp32p4/"
  click CTS2 "https://github.com/espressif/esp-sr/tree/master/esp-tts/esp_tts_chinese/esp32s2/"
  click CTS3 "https://github.com/espressif/esp-sr/tree/master/esp-tts/esp_tts_chinese/esp32s3/"
  click CTInc "https://github.com/espressif/esp-sr/tree/master/esp-tts/esp_tts_chinese/include/"
  click MN "https://github.com/espressif/esp-sr/tree/master/model/multinet_model/"
  click VN "https://github.com/espressif/esp-sr/tree/master/model/vadnet_model/"
  click WN "https://github.com/espressif/esp-sr/tree/master/model/wakenet_model/"
  click NS "https://github.com/espressif/esp-sr/tree/master/model/nsnet_model/"
  click Target "https://github.com/espressif/esp-sr/tree/master/model/target/"
  click Pack "https://github.com/espressif/esp-sr/blob/master/model/pack_model.py"
  click PrepFST "https://github.com/espressif/esp-sr/blob/master/tool/prepare_for_fst.py"
  click G2P "https://github.com/espressif/esp-sr/blob/master/tool/multinet_g2p.py"
  click Pinyin "https://github.com/espressif/esp-sr/blob/master/tool/multinet_pinyin.py"
  click BPE "https://github.com/espressif/esp-sr/blob/master/tool/fst/bpe.model"
  click Cmds "https://github.com/espressif/esp-sr/blob/master/tool/fst/commands_list.txt"
  click PrepFST2 "https://github.com/espressif/esp-sr/blob/master/tool/fst/prepare_for_fst.py"
  click GH "https://github.com/espressif/esp-sr/tree/master/.github/workflows/"
  click GL "https://github.com/espressif/esp-sr/blob/master/.gitlab-ci.yml"
  click Utils "https://github.com/espressif/esp-sr/blob/master/ci/utils.sh"
  click PytestIni "https://github.com/espressif/esp-sr/blob/master/pytest.ini"
  click Conftest "https://github.com/espressif/esp-sr/blob/master/conftest.py"
  click Docs "https://github.com/espressif/esp-sr/tree/master/docs/"
  click Conf "https://github.com/espressif/esp-sr/blob/master/docs/conf_common.py"
  click EN "https://github.com/espressif/esp-sr/tree/master/docs/en/"
  click ZH "https://github.com/espressif/esp-sr/tree/master/docs/zh_CN/"
  click CheckChars "https://github.com/espressif/esp-sr/blob/master/docs/check_doc_chars.py"
  click Sync "https://github.com/espressif/esp-sr/blob/master/docs/check_lang_folder_sync.sh"
  click Req "https://github.com/espressif/esp-sr/blob/master/docs/requirements.txt"
  click Chart "https://github.com/espressif/esp-sr/blob/master/docs/generate_chart.py"
  click AppSR "https://github.com/espressif/esp-sr/tree/master/test_apps/esp-sr/"
  click AppTTS "https://github.com/espressif/esp-sr/tree/master/test_apps/esp-tts/"
  click AppC5 "https://github.com/espressif/esp-sr/tree/master/test_apps/esp32c5/"

  %% Styles
  classDef hw fill:#DDDDDD,stroke:#888888
  classDef component fill:#A8D0E6,stroke:#2980B9
  classDef model fill:#A8E6A1,stroke:#27AE60
  classDef tools fill:#F9E79F,stroke:#B7950B
  classDef docs fill:#D7BDE2,stroke:#7D3C98
  classDef app fill:#F5B041,stroke:#D35400
  classDef runtime fill:#F1948A,stroke:#C0392B
```
