# huang-junsen/py-xiaozhi architecture

> Architecture diagram of the GitHub repository huang-junsen/py-xiaozhi, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-04-01). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/huang-junsen/py-xiaozhi
- Repository: https://github.com/huang-junsen/py-xiaozhi

## Overview

##### Project Type & Purpose
- This is a full-stack desktop (and possibly embedded) voice AI client. Its purpose is to simulate a smart voice assistant that supports voice commands, wake-word detection, GUI/CLI presentation, IoT integrations (e.g., camera, lamp, speaker, music player), and secure audio transmission.
- The README emphasizes voice AI, IoT control, and multi-modal interaction. The project supports both command-line and graphical interfaces and various communication protocols including MQTT and WebSocket.

##### File Structure Analysis
- The top level shows configuration files (e.g., config.json, camera config), documentation directories, hooks for packaging, scripts for build and device detection, and the primary source code in the “src” folder.
- Within the “src” folder, you have further directories that clearly separate concerns:
- audio_codecs – audio encoding/decoding.
- audio_processing – includes wake-word detection and VAD (Voice Activity Detection).
- constants – configuration constants.
- – separate implementations for CLI and GUI displays.
- iot – natural segregation of IoT devices including different “things” (like CameraVL, lamp, music_player, speaker, etc.) and a device management component.
- protocols – for MQTT and WebSocket communication.
- utils – common utility classes for logging, configuration, system info, and volume control.
- application.py – the core where the main application logic and orchestration happens.
- Supporting files include the main.py entrypoint, multiple requirements files for dependency management, and build scripts for packaging.

a. Main Components:
- Input/Audio Module: Covers audio capturing, wake-word detection (src/audio_processing/wake_word_detect.py) and VAD detection (src/audio_processing/vad_detector.py).
- Audio Codec Module: (src/audio_codecs) to handle audio encoding/decoding.
- Application Core: (src/application.py) orchestrates high-level interactions.
- Components: Both CLI (src/display/cli_display.py) and GUI (src/display/gui_display.py) for user interface.
- IoT Devices & Manager: (src/iot/thing.py, src/iot/thing_manager.py, and individual device implementations under src/iot/things) that control cameras, lamps, speakers, etc.
- Communication Protocols: (src/protocols) showing how the app interacts using MQTT and WebSocket.
- Utilities & Configurations: (src/utils) for config management, logging, system info, and volume control.
- External Dependencies: Hooks and libraries (e.g., opus.dll under libs/windows) and PyInstaller hooks for packaging.

b. Relationships & Interactions:
- Data Flow should be shown from the audio input module to audio processing (wake-word detection) then to the application core.
- The Application Core then routes commands to the appropriate display module, IoT device manager, or communication module based on user input and sensor data.
- Additionally, communication protocols should be illustrated as integration points between the application and external networks/services (for audio streaming, secure transmission, and remote controls).

c. Architectural Patterns & Design Principles:
- Modularity and separation of concerns. The project clearly divides responsibilities among its modules.
- Mention the use of a “manager” (IoT device manager) for orchestrating device interactions.
- Demonstrate abstraction layers – for instance, protocol classes abstract specific details of MQTT vs. WebSocket.
- Design pattern used for configuration management and logging via utility classes.

d. Technologies, Frameworks, or Libraries:
- Python as the main programming language.
- Audio processing libraries (like Vosk for speech recognition, as suggested by hook-vosk.py).
- Communication protocols libraries for MQTT and WebSocket.

## Diagram (Mermaid)

```mermaid
flowchart TD
    %% Input & Audio Processing Layer
    subgraph "Audio Input & Processing"
        A1["Wake-Word Detection"]:::input
        A2["VAD Detection"]:::input
        A3["Audio Codec Module"]:::input
    end

    %% Application Core
    subgraph "Application Core"
        B["Main Orchestrator"]:::core
    end

    %% Display Components
    subgraph "Display Interfaces"
        C1["CLI Display"]:::display
        C2["GUI Display"]:::display
        C3["Base Display"]:::display
    end

    %% IoT Devices & Manager
    subgraph "IoT Devices & Manager"
        D1["Device Abstraction"]:::iot
        D2["IoT Device Manager"]:::iot
        D3["Camera"]:::iot
        D4["Lamp"]:::iot
        D5["Music Player"]:::iot
        D6["Speaker"]:::iot
        D7["Query Bridge"]:::iot
    end

    %% Communication Protocols
    subgraph "Communication Protocols"
        E1["Protocol Base"]:::protocol
        E2["MQTT Protocol"]:::protocol
        E3["WebSocket Protocol"]:::protocol
    end

    %% Utilities & Configurations
    subgraph "Utilities & Configurations"
        F1["Config Management"]:::util
        F2["Logging Config"]:::util
        F3["System Info"]:::util
        F4["TTS Utility"]:::util
        F5["Volume Control"]:::util
    end

    %% External Dependencies & Packaging
    subgraph "Packaging & External Dependencies"
        G1["Vosk Hook"]:::external
        G2["Opus Hook"]:::external
        G3["Runtime Hook"]:::external
        G4["Windows Libraries"]:::external
    end

    %% State Transition (Inset Diagram)
    subgraph "状态流转图"
        S1["IDLE"]:::state
        S2["CONNECTING"]:::state
        S3["LISTENING"]:::state
        S4["SPEAKING"]:::state
        S1 -->|"to"| S2
        S2 -->|"to"| S3
        S3 -->|"to"| S4
        S4 -->|"to"| S1
    end

    %% Relationships

    %% Audio Processing to Application Core
    A1 -->|"audio_input"| B
    A2 -->|"audio_input"| B
    A3 -->|"codec_output"| B

    %% Core to Display Interfaces
    B -->|"display_command"| C1
    B -->|"display_command"| C2
    B -->|"display_command"| C3

    %% Core <-> IoT Devices
    B <-->|"iot_control"| D2
    D2 -->|"manage"| D1
    D2 -->|"command"| D3
    D2 -->|"command"| D4
    D2 -->|"command"| D5
    D2 -->|"command"| D6
    D2 -->|"command"| D7

    %% Core <-> Communication Protocols
    B <-->|"network_comm"| E1
    E1 -->|"mqtt_handling"| E2
    E1 -->|"websocket_handling"| E3

    %% Utilities interaction with Core
    F1 ---|"config"| B
    F2 ---|"logging"| B
    F3 ---|"sys_info"| B
    F4 ---|"tts"| B
    F5 ---|"volume"| B

    %% External Dependencies linked to Core
    G1 ---|"hook"| B
    G2 ---|"hook"| B
    G3 ---|"hook"| B
    G4 ---|"libs"| B

    %% Click Events for Input/Audio Module
    click A1 "https://github.com/huang-junsen/py-xiaozhi/blob/main/src/audio_processing/wake_word_detect.py"
    click A2 "https://github.com/huang-junsen/py-xiaozhi/blob/main/src/audio_processing/vad_detector.py"
    click A3 "https://github.com/huang-junsen/py-xiaozhi/tree/main/src/audio_codecs"

    %% Click Events for Application Core
    click B "https://github.com/huang-junsen/py-xiaozhi/blob/main/src/application.py"

    %% Click Events for Display Components
    click C1 "https://github.com/huang-junsen/py-xiaozhi/blob/main/src/display/cli_display.py"
    click C2 "https://github.com/huang-junsen/py-xiaozhi/blob/main/src/display/gui_display.py"
    click C3 "https://github.com/huang-junsen/py-xiaozhi/blob/main/src/display/base_display.py"

    %% Click Events for IoT Devices & Manager
    click D1 "https://github.com/huang-junsen/py-xiaozhi/blob/main/src/iot/thing.py"
    click D2 "https://github.com/huang-junsen/py-xiaozhi/blob/main/src/iot/thing_manager.py"
    click D3 "https://github.com/huang-junsen/py-xiaozhi/blob/main/src/iot/things/CameraVL/Camera.py"
    click D3 "https://github.com/huang-junsen/py-xiaozhi/blob/main/src/iot/things/CameraVL/VL.py"
    click D4 "https://github.com/huang-junsen/py-xiaozhi/blob/main/src/iot/things/lamp.py"
    click D5 "https://github.com/huang-junsen/py-xiaozhi/blob/main/src/iot/things/music_player.py"
    click D6 "https://github.com/huang-junsen/py-xiaozhi/blob/main/src/iot/things/speaker.py"
    click D7 "https://github.com/huang-junsen/py-xiaozhi/blob/main/src/iot/things/query_bridge_rag.py"

    %% Click Events for Communication Protocols
    click E1 "https://github.com/huang-junsen/py-xiaozhi/blob/main/src/protocols/protocol.py"
    click E2 "https://github.com/huang-junsen/py-xiaozhi/blob/main/src/protocols/mqtt_protocol.py"
    click E3 "https://github.com/huang-junsen/py-xiaozhi/blob/main/src/protocols/websocket_protocol.py"

    %% Click Events for Utilities & Configurations
    click F1 "https://github.com/huang-junsen/py-xiaozhi/blob/main/src/utils/config_manager.py"
    click F2 "https://github.com/huang-junsen/py-xiaozhi/blob/main/src/utils/logging_config.py"
    click F3 "https://github.com/huang-junsen/py-xiaozhi/blob/main/src/utils/system_info.py"
    click F4 "https://github.com/huang-junsen/py-xiaozhi/blob/main/src/utils/tts_utility.py"
    click F5 "https://github.com/huang-junsen/py-xiaozhi/blob/main/src/utils/volume_controller.py"

    %% Click Events for External Dependencies & Packaging
    click G1 "https://github.com/huang-junsen/py-xiaozhi/blob/main/hooks/hook-vosk.py"
    click G2 "https://github.com/huang-junsen/py-xiaozhi/blob/main/hooks/hook-opuslib.py"
    click G3 "https://github.com/huang-junsen/py-xiaozhi/blob/main/hooks/runtime_hook.py"
    click G4 "https://github.com/huang-junsen/py-xiaozhi/tree/main/libs/windows"

    %% Styles
    classDef input fill:#f9f,stroke:#333,stroke-width:2px;
    classDef core fill:#bbf,stroke:#333,stroke-width:2px;
    classDef display fill:#cfc,stroke:#333,stroke-width:2px;
    classDef iot fill:#fc9,stroke:#333,stroke-width:2px;
    classDef protocol fill:#9cf,stroke:#333,stroke-width:2px;
    classDef util fill:#ff9,stroke:#333,stroke-width:2px;
    classDef external fill:#ccc,stroke:#333,stroke-width:2px;
    classDef state fill:#ddf,stroke:#333,stroke-width:2px;
```
