# 78/xiaozhi-esp32 architecture

> Architecture diagram of the GitHub repository 78/xiaozhi-esp32, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-10-19). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/78/xiaozhi-esp32
- Repository: https://github.com/78/xiaozhi-esp32

## Overview

XiaoZhi is embedded voice-chatbot firmware: a user speaks to a supported device, which captures and processes audio, communicates with a large-model service, and presents spoken or visual responses. Device-side MCP tools also let models control hardware. The graph combines the sampled application/audio calls and SOURCE INDEX references with README-documented capabilities; unsampled display, camera, networking, and provisioning internals are represented at their supported source paths, without asserting their internal call chains.

Main entry points: [`main/application.cc`](https://github.com/78/xiaozhi-esp32/blob/main/main/application.cc) (Application loop), [`main/boards/common/board.h`](https://github.com/78/xiaozhi-esp32/blob/main/main/boards/common/board.h) (Board hardware), [`main/audio/audio_service.cc`](https://github.com/78/xiaozhi-esp32/blob/main/main/audio/audio_service.cc) (Audio service), [`main/audio/engines`](https://github.com/78/xiaozhi-esp32/tree/main/main/audio/engines) (Audio engines)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_device["Device platform"]
  node_board["Board hardware<br/>[board.h]"]
  node_board_impls["Board variants"]
  node_robot_tools["Robot actions<br/>[otto_controller.cc]"]
  node_display["Display UI<br/>[lcd_display.cc]"]
  node_camera["Camera input<br/>[camera.h]"]
  node_assets["Custom assets<br/>[assets.cc]"]
  node_settings[("Device settings<br/>[settings.h]")]
end

subgraph group_audio["Voice processing"]
  node_codec["Audio codec<br/>[audio_codec.h]"]
  node_audio_service["Audio service<br/>[audio_service.cc]"]
  node_audio_engines["Audio engines"]
  node_wake_words["Wake words"]
  node_ogg["Ogg demuxer<br/>[ogg_demuxer.cc]"]
end

subgraph group_interaction["Conversation and control"]
  node_app["Application loop<br/>[application.cc]"]
  node_mcp["Device MCP tools<br/>[mcp_server.cc]"]
end

subgraph group_connectivity["Networking and services"]
  node_protocol["Protocol interface<br/>[protocol.h]"]
  node_mqtt["MQTT and UDP<br/>[mqtt_protocol.cc]"]
  node_websocket["WebSocket"]
  node_network["Network interface<br/>[board.h]"]
end

node_user(("Device user"))
node_model_service{{"Model service"}}

node_user -->|"speaks to"| node_board
node_app -->|"gets hardware"| node_board
node_app -->|"sets up UI"| node_display
node_app -->|"initializes audio"| node_audio_service
node_board -->|"provides codec"| node_codec
node_board_impls -.->|"implements boards"| node_board
node_audio_service -->|"uses codec"| node_codec
node_audio_service -->|"uses engines"| node_audio_engines
node_audio_engines -->|"processes audio"| node_codec
node_audio_engines -->|"detects wake words"| node_wake_words
node_audio_service -->|"demuxes audio"| node_ogg
node_user -.->|"provides vision input"| node_camera
node_app -->|"initializes protocol"| node_protocol
node_protocol -->|"implements transport"| node_mqtt
node_protocol -.->|"implements transport"| node_websocket
node_mqtt -.->|"exchanges messages"| node_model_service
node_websocket -.->|"exchanges messages"| node_model_service
node_app -->|"uses MCP server"| node_mcp
node_mcp -->|"dispatches actions"| node_robot_tools
node_model_service -.->|"invokes device tools"| node_mcp
node_display -.->|"shows expressions"| node_assets
node_board -->|"provides network"| node_network
node_app -.->|"uses configuration"| node_settings

click node_app "https://github.com/78/xiaozhi-esp32/blob/main/main/application.cc"
click node_board "https://github.com/78/xiaozhi-esp32/blob/main/main/boards/common/board.h"
click node_board_impls "https://github.com/78/xiaozhi-esp32/tree/main/main/boards"
click node_codec "https://github.com/78/xiaozhi-esp32/blob/main/main/audio/audio_codec.h"
click node_audio_service "https://github.com/78/xiaozhi-esp32/blob/main/main/audio/audio_service.cc"
click node_audio_engines "https://github.com/78/xiaozhi-esp32/tree/main/main/audio/engines"
click node_wake_words "https://github.com/78/xiaozhi-esp32/tree/main/main/audio/wake_words"
click node_ogg "https://github.com/78/xiaozhi-esp32/blob/main/main/audio/demuxer/ogg_demuxer.cc"
click node_protocol "https://github.com/78/xiaozhi-esp32/blob/main/main/protocols/protocol.h"
click node_mqtt "https://github.com/78/xiaozhi-esp32/blob/main/main/protocols/mqtt_protocol.cc"
click node_websocket "https://github.com/78/xiaozhi-esp32/blob/main/main/protocols/websocket_protocol.cc"
click node_mcp "https://github.com/78/xiaozhi-esp32/blob/main/main/mcp_server.cc"
click node_robot_tools "https://github.com/78/xiaozhi-esp32/blob/main/main/boards/otto-robot/otto_controller.cc"
click node_display "https://github.com/78/xiaozhi-esp32/blob/main/main/display/lcd_display.cc"
click node_camera "https://github.com/78/xiaozhi-esp32/blob/main/main/boards/common/camera.h"
click node_assets "https://github.com/78/xiaozhi-esp32/blob/main/main/assets.cc"
click node_settings "https://github.com/78/xiaozhi-esp32/blob/main/main/settings.h"
click node_network "https://github.com/78/xiaozhi-esp32/blob/main/main/boards/common/board.h"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_board,node_board_impls,node_robot_tools,node_display,node_camera,node_assets,node_settings,node_user toneBlue
class node_codec,node_audio_service,node_audio_engines,node_wake_words,node_ogg toneAmber
class node_app,node_mcp toneMint
class node_protocol,node_mqtt,node_websocket,node_network toneRose
class node_model_service toneIndigo
```

## Components

### Device platform

- [Board hardware](https://github.com/78/xiaozhi-esp32/blob/main/main/boards/common/board.h): `main/boards/common/board.h`
- [Board variants](https://github.com/78/xiaozhi-esp32/tree/main/main/boards): `main/boards`
- [Robot actions](https://github.com/78/xiaozhi-esp32/blob/main/main/boards/otto-robot/otto_controller.cc): `main/boards/otto-robot/otto_controller.cc`
- [Display UI](https://github.com/78/xiaozhi-esp32/blob/main/main/display/lcd_display.cc): `main/display/lcd_display.cc`
- [Camera input](https://github.com/78/xiaozhi-esp32/blob/main/main/boards/common/camera.h): `main/boards/common/camera.h`
- [Custom assets](https://github.com/78/xiaozhi-esp32/blob/main/main/assets.cc): `main/assets.cc`
- [Device settings](https://github.com/78/xiaozhi-esp32/blob/main/main/settings.h): `main/settings.h`

### Voice processing

- [Audio codec](https://github.com/78/xiaozhi-esp32/blob/main/main/audio/audio_codec.h): `main/audio/audio_codec.h`
- [Audio service](https://github.com/78/xiaozhi-esp32/blob/main/main/audio/audio_service.cc): `main/audio/audio_service.cc`
- [Audio engines](https://github.com/78/xiaozhi-esp32/tree/main/main/audio/engines): `main/audio/engines`
- [Wake words](https://github.com/78/xiaozhi-esp32/tree/main/main/audio/wake_words): `main/audio/wake_words`
- [Ogg demuxer](https://github.com/78/xiaozhi-esp32/blob/main/main/audio/demuxer/ogg_demuxer.cc): `main/audio/demuxer/ogg_demuxer.cc`

### Conversation and control

- [Application loop](https://github.com/78/xiaozhi-esp32/blob/main/main/application.cc): `main/application.cc`
- [Device MCP tools](https://github.com/78/xiaozhi-esp32/blob/main/main/mcp_server.cc): `main/mcp_server.cc`

### Networking and services

- [Protocol interface](https://github.com/78/xiaozhi-esp32/blob/main/main/protocols/protocol.h): `main/protocols/protocol.h`
- [MQTT and UDP](https://github.com/78/xiaozhi-esp32/blob/main/main/protocols/mqtt_protocol.cc): `main/protocols/mqtt_protocol.cc`
- [WebSocket](https://github.com/78/xiaozhi-esp32/blob/main/main/protocols/websocket_protocol.cc): `main/protocols/websocket_protocol.cc`
- [Network interface](https://github.com/78/xiaozhi-esp32/blob/main/main/boards/common/board.h): `main/boards/common/board.h`

### Other components

- **Device user**
- **Model service**

## Connections

- Device user → Board hardware: speaks to
- Application loop → Board hardware: gets hardware (evidence: `main/application.cc`)
- Application loop → Display UI: sets up UI (evidence: `main/application.cc`)
- Application loop → Audio service: initializes audio (evidence: `main/application.cc`)
- Board hardware → Audio codec: provides codec (evidence: `main/boards/common/board.h`)
- Board variants → Board hardware: implements boards (evidence: `main/boards/otto-robot/otto_controller.cc`)
- Audio service → Audio codec: uses codec
- Audio service → Audio engines: uses engines (evidence: `main/audio/audio_service.cc`)
- Audio engines → Audio codec: processes audio
- Audio engines → Wake words: detects wake words (evidence: `main/audio/engines/afe_audio_engine.cc`)
- Audio service → Ogg demuxer: demuxes audio
- Device user → Camera input: provides vision input
- Application loop → Protocol interface: initializes protocol (evidence: `main/application.cc`)
- Protocol interface → MQTT and UDP: implements transport
- Protocol interface → WebSocket: implements transport
- MQTT and UDP → Model service: exchanges messages
- WebSocket → Model service: exchanges messages
- Application loop → Device MCP tools: uses MCP server (evidence: `main/application.cc`)
- Device MCP tools → Robot actions: dispatches actions (evidence: `main/boards/otto-robot/otto_controller.cc`)
- Model service → Device MCP tools: invokes device tools
- Display UI → Custom assets: shows expressions
- Board hardware → Network interface: provides network (evidence: `main/boards/common/board.h`)
- Application loop → Device settings: uses configuration (evidence: `main/application.cc`)
