How ownding/open-xiaoai works
Open-XiaoAI modifies Xiaomi smart speakers so their microphone/audio and device controls can participate in AI voice workflows. The principal path is a speaker running the Rust client, which exchanges messages with a voice service; optional integrations add Hermes reasoning and Home Assistant control, while Sherpa provides an alternate TTS deployment. The repository also includes firmware-patch tooling and a separate promo-video production project. Internal edges are limited to relationships shown in excerpts or the source index; unsampled README-described capabilities are represented without inferred wiring.
Main entry points: message.rs WebSocket Transport · handler.rs Message Dispatch · hermes.py XiaoZhi Service · rpc.rs RPC
Components 30
Speaker Client
- Client Runtime
packages/client-rust/src/bin/client.rs - Audio Capture
packages/client-rust/src/services/audio/record.rs - Audio Playback
packages/client-rust/src/services/audio/play.rs - Audio Config
packages/client-rust/src/services/audio/config.rs - WebSocket Transport
packages/client-rust/src/services/connect/message.rs - Message Dispatch
packages/client-rust/src/services/connect/handler.rs - RPC
packages/client-rust/src/services/connect/rpc.rs - Message Types
packages/client-rust/src/services/connect/data.rs - Speaker Control
packages/client-rust/src/services/speaker.rs - Shell Utilities
packages/client-rust/src/utils/shell.rs - OTA Patch Tool
packages/client-patch/src/ota.ts
Voice Integrations
- XiaoZhi Service
deploy/hermes/xiaozhi-provider/hermes.py - Hermes Provider
deploy/hermes/xiaozhi-provider/hermes.py - Sherpa TTS
deploy/sherpa-tts/server.py - TTS Mode Switching
deploy/sherpa-tts/switch_config.py - LLM Switching
deploy/hermes/switch_llm.py
Home Automation
- Home Assistant
- HA Onboarding
deploy/hermes/demo-ha/onboard.py - Device Mapping
deploy/hermes/ha_device_table.py
Promo Production
- Video Timeline
promo-video/src/timeline.py - 3D Scene
promo-video/src/blender/scene.py - Shot Plates
promo-video/src/blender/plates.py - Plate Renderer
promo-video/src/blender/render_plate.py - Audio Synth
promo-video/src/audio/synth.py - Music Score
promo-video/src/audio/score.py - Audio Mix
promo-video/src/audio/mix.py - Video Compositor
promo-video/src/compositor/comp.js - Render Entry
promo-video/src/compositor/render.mjs
Other components
- Speaker User
- Xiaomi Speaker
Connections 27
- Speaker User → Xiaomi Speaker: speaks to
README.md - Xiaomi Speaker → Client Runtime: runs client
README.md - Client Runtime → WebSocket Transport: uses transport
- Audio Capture → Audio Config: reads config
packages/client-rust/src/services/audio/record.rs - Audio Playback → Audio Config: reads config
packages/client-rust/src/services/audio/play.rs - WebSocket Transport → Message Types: serializes messages
packages/client-rust/src/services/connect/message.rs - WebSocket Transport → Message Dispatch: dispatches messages
packages/client-rust/src/services/connect/message.rs - WebSocket Transport → RPC: sends requests
packages/client-rust/src/services/connect/message.rs - Message Dispatch → RPC: routes requests
packages/client-rust/src/services/connect/handler.rs - Message Dispatch → WebSocket Transport: sends responses
packages/client-rust/src/services/connect/handler.rs - Speaker Control → RPC: calls remote
packages/client-rust/src/services/speaker.rs - Speaker Control → Shell Utilities: uses commands
packages/client-rust/src/services/speaker.rs - Xiaomi Speaker → XiaoZhi Service: streams voice
README.md - XiaoZhi Service → Hermes Provider: delegates reasoning
README.md - Hermes Provider → Home Assistant: controls devices
README.md - HA Onboarding → Home Assistant: onboards
- Device Mapping → Home Assistant: maps devices
- XiaoZhi Service → Sherpa TTS: uses alternate TTS
- TTS Mode Switching → Sherpa TTS: switches mode
- LLM Switching → Hermes Provider: selects provider
- OTA Patch Tool → Xiaomi Speaker: updates firmware
- Plate Renderer → Shot Plates: renders plates
promo-video/src/blender/render_plate.py - Plate Renderer → 3D Scene: uses scene
promo-video/src/blender/render_plate.py - Shot Plates → 3D Scene: stages shots
promo-video/src/blender/plates.py - Music Score → Audio Synth: uses synthesis
promo-video/src/audio/score.py - Audio Mix → Audio Synth: uses synthesis
promo-video/src/audio/mix.py - Audio Mix → Music Score: mixes score
promo-video/src/audio/mix.py