How 78/xiaozhi-esp32 works
XiaoZhi is embedded voice-chatbot firmware: a user speaks to a supported device, which captures and processes audio, communicates with a large-model service, and presents spoken or visual responses. Device-side MCP tools also let models control hardware. The graph combines the sampled application/audio calls and SOURCE INDEX references with README-documented capabilities; unsampled display, camera, networking, and provisioning internals are represented at their supported source paths, without asserting their internal call chains.
Main entry points: application.cc Application loop · board.h Board hardware · audio_service.cc Audio service · engines Audio engines
Components 20
Device platform
- Board hardware
main/boards/common/board.h - Board variants
main/boards - Robot actions
main/boards/otto-robot/otto_controller.cc - Display UI
main/display/lcd_display.cc - Camera input
main/boards/common/camera.h - Custom assets
main/assets.cc - Device settings
main/settings.h
Voice processing
- Audio codec
main/audio/audio_codec.h - Audio service
main/audio/audio_service.cc - Audio engines
main/audio/engines - Wake words
main/audio/wake_words - Ogg demuxer
main/audio/demuxer/ogg_demuxer.cc
Conversation and control
- Application loop
main/application.cc - Device MCP tools
main/mcp_server.cc
Networking and services
- Protocol interface
main/protocols/protocol.h - MQTT and UDP
main/protocols/mqtt_protocol.cc - WebSocket
main/protocols/websocket_protocol.cc - Network interface
main/boards/common/board.h
Other components
- Device user
- Model service
Connections 23
- Device user → Board hardware: speaks to
- Application loop → Board hardware: gets hardware
main/application.cc - Application loop → Display UI: sets up UI
main/application.cc - Application loop → Audio service: initializes audio
main/application.cc - Board hardware → Audio codec: provides codec
main/boards/common/board.h - Board variants → Board hardware: implements boards
main/boards/otto-robot/otto_controller.cc - Audio service → Audio codec: uses codec
- Audio service → Audio engines: uses engines
main/audio/audio_service.cc - Audio engines → Audio codec: processes audio
- Audio engines → Wake words: detects wake words
main/audio/engines/afe_audio_engine.cc - Audio service → Ogg demuxer: demuxes audio
- Device user → Camera input: provides vision input
- Application loop → Protocol interface: initializes protocol
main/application.cc - Protocol interface → MQTT and UDP: implements transport
- Protocol interface → WebSocket: implements transport
- MQTT and UDP → Model service: exchanges messages
- WebSocket → Model service: exchanges messages
- Application loop → Device MCP tools: uses MCP server
main/application.cc - Device MCP tools → Robot actions: dispatches actions
main/boards/otto-robot/otto_controller.cc - Model service → Device MCP tools: invokes device tools
- Display UI → Custom assets: shows expressions
- Board hardware → Network interface: provides network
main/boards/common/board.h - Application loop → Device settings: uses configuration
main/application.cc