# dhruthi9701/speech-recognition-with-esp32-using-vosk-model architecture

> Architecture diagram of the GitHub repository dhruthi9701/speech-recognition-with-esp32-using-vosk-model, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-06-16). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/dhruthi9701/speech-recognition-with-esp32-using-vosk-model
- Repository: https://github.com/dhruthi9701/speech-recognition-with-esp32-using-vosk-model

## Overview

##### Project Type & Purpose
- This is a hybrid embedded/host‐based system:
– Host side: Python wake-word listener using Vosk.
– Embedded side: ESP32 + DFPlayer Mini handle MP3 playback and power management.
- Purpose: Listen for keywords (“jarvis”, “hello”, etc.) on a PC microphone, send a trigger byte over serial to the ESP32, play a matching MP3 from microSD, optionally deep-sleep.

##### Main Components
1. Hardware
– Microphone (PC-side)
– PC (runs Python + Vosk model)
– USB/Serial link
– ESP32 Dev Board (UART2 on GPIO18/19)
– DFPlayer Mini MP3 Module
– microSD Card (contains 0001.mp3–0005.mp3)
– Speaker
2. Software & Firmware
– Python scripts:
- mic_list.py, mic_rms.py, test_serial.py (utilities)
- wakeword_vosk_to_esp32.py (core listener & serial sender)
- Vosk model folder (vosk-model-small-en-us-0.15)
– Arduino sketch (esp32_to_dfplayer.ino) using DFRobotDFPlayerMini library

##### Relationships & Data Flows
1. Audio Input → Vosk
- Mic captures live audio → sounddevice → Vosk model → triggers keyword detection
2. Trigger Byte over Serial
- Python sends one of {0x01…0x05} via pyserial to ESP32 USB port
3. ESP32 Command Handling
- UART2 RX receives byte → switch statement → DFRobotDFPlayerMini.play(trackIndex)
- If byte==0x05, play track 5 then call esp_deep_sleep()
4. Playback
- DFPlayer reads MP3 from microSD → outputs audio to speaker
5. Deep-Sleep (on “hammer time”)
- ESP32 enters low-power mode after playing track 5

##### Architectural Patterns & Principles
- Clear separation of concerns:
– PC host: compute-heavy speech recognition
– Embedded: real-time playback and power management
- Event-driven control flow (keyword → trigger → action)
- Serial (UART) as boundary/interface between host and device

##### Technologies & Frameworks
- Python 3 with vosk, sounddevice, pyserial
- Vosk offline speech model
- Arduino framework for ESP32
- DFRobotDFPlayerMini Arduino library

– Note on deep-sleep behavior

## Diagram (Mermaid)

```mermaid
flowchart TD
    %% Host Side
    subgraph "PC Host" 
        direction TB
        Mic["Microphone"]:::peripheral
        SoundDevice["sounddevice (Python)"]:::host
        Wakeword["wakeword_vosk_to_esp32.py"]:::host
        VoskModel["Vosk Model"]:::storage
        PySerial["pyserial"]:::host
        Utils_sub["Utility Scripts"]:::doc
        mic_list["mic_list.py"]:::doc
        mic_rms["mic_rms.py"]:::doc
        test_serial["test_serial.py"]:::doc
        README["README.md"]:::doc
        Mic -->|"PCM audio"| SoundDevice
        SoundDevice -->|"audio stream"| Wakeword
        Wakeword -->|"uses model"| VoskModel
        Wakeword -->|"Trigger byte 0x01…0x05"| PySerial
        Utils_sub --> mic_list
        Utils_sub --> mic_rms
        Utils_sub --> test_serial
        Utils_sub --> README
    end

    %% Serial Link
    PySerial -->|"Serial USB"| ESP32_UART

    %% Embedded Side
    subgraph "ESP32 Embedded" 
        direction TB
        ESP32_UART["ESP32 UART2 RX"]:::embedded
        Firmware["esp32_to_dfplayer.ino"]:::embedded
        DFPlayer["DFPlayer Mini"]:::peripheral
        MicroSD["MicroSD Card"]:::storage
        Speaker["Speaker"]:::peripheral
        DeepSleep["Deep Sleep Mode"]:::embedded
        ESP32_UART -->|"byte received"| Firmware
        Firmware -->|"play(trackIndex)"| DFPlayer
        DFPlayer -->|"reads MP3"| MicroSD
        DFPlayer -->|"audio out"| Speaker
        Firmware -->|"{if track5}"| DeepSleep
    end

    %% Click Events
    click Wakeword "https://github.com/dhruthi9701/speech-recognition-with-esp32-using-vosk-model/blob/main/esp32_to_py/wakeword_vosk_to_esp32.py"
    click mic_list "https://github.com/dhruthi9701/speech-recognition-with-esp32-using-vosk-model/blob/main/esp32_to_py/mic_list.py"
    click mic_rms "https://github.com/dhruthi9701/speech-recognition-with-esp32-using-vosk-model/blob/main/esp32_to_py/mic_rms.py"
    click test_serial "https://github.com/dhruthi9701/speech-recognition-with-esp32-using-vosk-model/blob/main/esp32_to_py/test_serial.py"
    click VoskModel "https://github.com/dhruthi9701/speech-recognition-with-esp32-using-vosk-model/tree/main/esp32_to_py/vosk-model-small-en-us-0.15/"
    click Firmware "https://github.com/dhruthi9701/speech-recognition-with-esp32-using-vosk-model/blob/main/esp32_to_dfplayer/esp32_to_dfplayer.ino"
    click README "https://github.com/dhruthi9701/speech-recognition-with-esp32-using-vosk-model/blob/main/README.md"

    %% Styles
    classDef host fill:#ADD8E6,stroke:#000,stroke-width:1px;
    classDef embedded fill:#90EE90,stroke:#000,stroke-width:1px;
    classDef peripheral fill:#D3D3D3,stroke:#000,stroke-width:1px;
    classDef storage fill:#FFEFD5,stroke:#000,stroke-width:1px,stroke-dasharray: 5 5;
    classDef doc fill:#F0E68C,stroke:#000,stroke-width:1px,stroke-dasharray: 2 2;
```
