# m5stack/stackchan architecture

> Architecture diagram of the GitHub repository m5stack/stackchan, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-07-03). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/m5stack/stackchan
- Repository: https://github.com/m5stack/stackchan

## Overview

1. **Repository shape**
   - This repo is a multi-target product stack for StackChan: a **Flutter mobile app**, **ESP32-S3 firmware** for the robot, a separate **ESP32 remote controller firmware**, and a **Go backend server**.
   - The architecture is split by runtime and deployment boundary, not by package alone.

2. **Core product boundary: the robot**
   - The main embedded system lives in `firmware/` and runs on the **M5Stack CoreS3 / ESP32-S3**.
   - It owns the physical robot: display, camera, audio, servos, IMU, LEDs, touch, RTC, BLE/Wi‑Fi, OTA, and ESP-NOW.
   - Its job is to turn hardware events and network commands into robot behavior and to expose device capabilities upward to the app/server.

3. **Firmware layering**
   - `firmware/main/hal/` is the hardware abstraction layer: audio, BLE, ESP-NOW, network, OTA, RTC, servo, IMU, camera/display, NFC-like peripherals, and board-specific glue.
   - `firmware/main/stackchan/` is the robot domain layer: avatar rendering, motion, animation, modifiers, JSON helpers, and utility math.
   - `firmware/main/apps/` is a mini app framework with separate user-facing modes such as launcher, setup, app center, AI agent, avatar, dance, and EZ data.

4. **Robot behavior and state model**
   - The firmware’s behavior is organized around **modifiers** and **motion/animation** primitives.
   - Avatar/emotion concepts are modeled separately from servo motion, which lets the robot express states like speaking, blinking, idle motion, dance, and head pet interactions.
   - This is a key diagram boundary: “behavior state” is distinct from “hardware actuation.”

5. **Mobile app as the orchestration client**
   - `app/lib/` is a **Flutter** client that coordinates setup, device binding, dance management, conversation, monitoring, music upload, and settings.
   - It talks to the backend over HTTP and to devices over network/socket utilities.
   - The app is not just a UI shell; it contains the product’s user flows for onboarding, pairing, content management, and live control.

6. **Mobile app native integration**
   - iOS has native views and bridging under `app/ios/Runner/`, including AR/3D robot views, motion screens, and a `NativeBridge`.
   - Android has a parallel native layer under `app/android/` with Kotlin models and a 3D model asset.
   - These native layers exist to render the robot locally, not merely to host Flutter UI.

7. **App-side domain and transport**
   - The Flutter models include StackChan/XiaoZhi concepts such as **device**, **dance**, **expression**, **agent**, **conversation**, **registration**, and **music info**.
   - Network code is separated into HTTP and WebSocket utilities, indicating a split between request/response APIs and live sessions.
   - Utility modules handle audio, Bluetooth, ML Kit, RSA, MAC validation, and native bridge calls, so the app also performs local device interaction and media handling.

8. **Backend server as shared control plane**
   - `server/` is a **Go** backend with a layered structure: API definitions, controllers, services, DAO/model layers, middleware, web socket support, and deployment manifests.
   - It exposes versioned endpoints for admin, app store, dance, device, file, friend, pano, post, stackchan device, user, and xiaozhi.
   - This server acts as the shared control plane for device registration/binding, content distribution, user accounts, and media/content management.

9. **Server responsibilities and data persistence**
   - Controllers map HTTP APIs to services; services encapsulate business logic; DAOs handle storage access through generated/internal model types.
   - The presence of `sqlite_sequence` entities and SQL setup scripts indicates relational persistence, while `server/file/` and `server/resource/` host static/media content.
   - There is also WebSocket support in `server/internal/web_socket/`, likely for live device or avatar session updates.

10. **Remote controller as a separate embedded client**
   - `remote/code/` is another **ESP-IDF/LVGL** application for an ESP32-based handheld/standalone controller.
   - It contains its own joystick handling, ESP-NOW initialization, and LVGL screens for setup, running, and IMU views.
   - This firmware is a direct peer of the robot, focused on local control rather than cloud orchestration.

11. **Primary data flows**
   - **User ↔ Flutter app ↔ backend server**: account/login, device binding, app store content, dance lists, files, and XiaoZhi token/license workflows.
   - **Flutter app ↔ robot firmware**: live control, monitoring, video/avatar display, configuration, and OTA-related coordination.
   - **Remote controller ↔ robot firmware**: low-latency ESP-NOW control for motion/interaction without going through the cloud.
   - **Robot firmware ↔ hardware**: servos, LEDs, sensors, camera, audio, and touch are managed locally through HAL and motion subsystems.

12. **Shared content and assets**
   - The repo includes common robot assets: 3D models (`.glb`, `.scn`), setup videos, icons, sounds, and UI artwork.
   - These assets are consumed across platforms: Flutter UI, iOS native views, Android resources, and embedded firmware assets.
   - This means the architecture has a “shared content” layer even though the runtimes differ.

13. **Deployment boundary**
   - The backend includes Docker/Kustomize manifests and config, implying containerized deployment.
   - The firmware side is built as embedded binaries with ESP-IDF and board-specific configuration.
   - The mobile app is separately packaged for iOS and Android with native platform projects under `app/ios` and `app/android`.

14. **What to draw in an architecture diagram**
   - Draw four main boxes: **mobile app**, **robot firmware**, **remote controller firmware**, and **backend server**.
   - Show the server as the persistent control/content plane, the robot firmware as the hardware/runtime plane, the mobile app as the user-facing orchestration plane, and the remote controller as a direct low-latency input plane.
   - Annotate the robot firmware with internal layers: **HAL → stackchan behavior engine → app modes**.

Main entry points: [`app/lib`](https://github.com/m5stack/stackchan/tree/main/app/lib) (Flutter app), [`firmware/main/hal`](https://github.com/m5stack/stackchan/tree/main/firmware/main/hal) (HAL layer), [`app/assets`](https://github.com/m5stack/stackchan/tree/main/app/assets) (Shared media), [`firmware/main/main.cpp`](https://github.com/m5stack/stackchan/blob/main/firmware/main/main.cpp) (Main app)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_mobile_app["Mobile app"]
  node_app_flutter["Flutter app<br/>UI/orchestration"]
  node_app_network["App transport<br/>HTTP/WebSocket"]
  node_app_domain["App models<br/>product domain"]
  node_app_native_ios["iOS native<br/>Swift bridge"]
  node_app_native_android["Android native<br/>Kotlin bridge"]
end

subgraph group_robot_firmware["Robot firmware"]
  node_firmware_main["Main app<br/>embedded entry<br/>[main.cpp]"]
  node_firmware_hal["HAL layer<br/>hardware abstraction"]
  node_firmware_behavior["Behavior engine<br/>stackchan domain"]
  node_firmware_apps["Robot apps<br/>mode framework"]
  node_firmware_assets["Firmware assets<br/>embedded media"]
end

subgraph group_remote_firmware["Remote controller"]
  node_remote_main["Remote app<br/>ESP-IDF entry"]
  node_remote_espnow["ESP-NOW link<br/>radio transport"]
  node_remote_ui["Remote UI<br/>LVGL screens"]
  node_remote_input["Joystick input<br/>controls"]
end

subgraph group_backend_server["Backend server"]
  node_server_api["API surface<br/>versioned endpoints"]
  node_server_handlers["Controllers<br/>HTTP handlers"]
  node_server_services["Services<br/>business logic"]
  node_server_data[("Data layer<br/>DAO/model")]
  node_server_ws["WebSocket hub<br/>live sessions"]
  node_server_deploy["Deploy manifests<br/>container/kustomize"]
end

subgraph group_shared_assets["Shared assets"]
  node_shared_media["Shared media<br/>cross-platform assets"]
end

node_app_flutter -->|"uses"| node_app_network
node_app_flutter -->|"binds"| node_app_domain
node_app_flutter -->|"bridges"| node_app_native_ios
node_app_flutter -->|"bridges"| node_app_native_android
node_app_network -->|"HTTP/WebSocket"| node_server_api
node_app_flutter -.->|"device control"| node_firmware_hal
node_firmware_main -->|"initializes"| node_firmware_hal
node_firmware_main -->|"runs"| node_firmware_behavior
node_firmware_main -->|"hosts"| node_firmware_apps
node_firmware_apps -->|"invokes"| node_firmware_behavior
node_firmware_behavior -->|"acts on"| node_firmware_hal
node_firmware_hal -->|"loads"| node_firmware_assets
node_remote_main -->|"reads"| node_remote_input
node_remote_main -->|"sends"| node_remote_espnow
node_remote_main -->|"renders"| node_remote_ui
node_remote_espnow -.->|"peer control"| node_firmware_hal
node_server_api -->|"dispatches"| node_server_handlers
node_server_handlers -->|"calls"| node_server_services
node_server_services -->|"persists"| node_server_data
node_server_handlers -->|"upgrades"| node_server_ws
node_server_api -.->|"deploys"| node_server_deploy
node_shared_media -->|"consumed by"| node_app_flutter
node_shared_media -->|"consumed by"| node_app_native_ios
node_shared_media -->|"consumed by"| node_app_native_android
node_shared_media -.->|"mirrored in"| node_firmware_assets
node_server_services -.->|"orchestrates"| node_firmware_main

click node_app_flutter "https://github.com/m5stack/stackchan/tree/main/app/lib"
click node_app_network "https://github.com/m5stack/stackchan/tree/main/app/lib/network"
click node_app_domain "https://github.com/m5stack/stackchan/tree/main/app/lib/model"
click node_app_native_ios "https://github.com/m5stack/stackchan/tree/main/app/ios/Runner"
click node_app_native_android "https://github.com/m5stack/stackchan/tree/main/app/android/app/src/main"
click node_firmware_main "https://github.com/m5stack/stackchan/blob/main/firmware/main/main.cpp"
click node_firmware_hal "https://github.com/m5stack/stackchan/tree/main/firmware/main/hal"
click node_firmware_behavior "https://github.com/m5stack/stackchan/tree/main/firmware/main/stackchan"
click node_firmware_apps "https://github.com/m5stack/stackchan/tree/main/firmware/main/apps"
click node_firmware_assets "https://github.com/m5stack/stackchan/tree/main/firmware/main/assets"
click node_remote_main "https://github.com/m5stack/stackchan/blob/main/remote/code/main/StackChan-RemoteControl-ESPNow.cpp"
click node_remote_espnow "https://github.com/m5stack/stackchan/tree/main/remote/code/main/esp_now"
click node_remote_ui "https://github.com/m5stack/stackchan/tree/main/remote/code/main/ui"
click node_remote_input "https://github.com/m5stack/stackchan/tree/main/remote/code/main/joystick"
click node_server_api "https://github.com/m5stack/stackchan/tree/main/server/api"
click node_server_handlers "https://github.com/m5stack/stackchan/tree/main/server/internal/controller"
click node_server_services "https://github.com/m5stack/stackchan/tree/main/server/internal/service"
click node_server_data "https://github.com/m5stack/stackchan/tree/main/server/internal/dao"
click node_server_ws "https://github.com/m5stack/stackchan/tree/main/server/internal/web_socket"
click node_server_deploy "https://github.com/m5stack/stackchan/tree/main/server/manifest"
click node_shared_media "https://github.com/m5stack/stackchan/tree/main/app/assets"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_app_flutter,node_app_network,node_app_domain,node_app_native_ios,node_app_native_android toneBlue
class node_firmware_main,node_firmware_hal,node_firmware_behavior,node_firmware_apps,node_firmware_assets toneAmber
class node_remote_main,node_remote_espnow,node_remote_ui,node_remote_input toneMint
class node_server_api,node_server_handlers,node_server_services,node_server_data,node_server_ws,node_server_deploy toneRose
class node_shared_media toneIndigo
```

## Components

### Mobile app

Flutter client plus native iOS/Android integrations.

- [Flutter app](https://github.com/m5stack/stackchan/tree/main/app/lib): UI/orchestration · Main cross-platform mobile client for setup, pairing, content, and live control. · `app/lib`
- [App transport](https://github.com/m5stack/stackchan/tree/main/app/lib/network): HTTP/WebSocket · Network layer separating request APIs from live sessions. · `app/lib/network`
- [App models](https://github.com/m5stack/stackchan/tree/main/app/lib/model): product domain · Device, dance, expression, agent, conversation, registration, and music data models. · `app/lib/model`
- [iOS native](https://github.com/m5stack/stackchan/tree/main/app/ios/Runner): Swift bridge · Native robot rendering and motion views plus bridge code for iOS. · `app/ios/Runner`
- [Android native](https://github.com/m5stack/stackchan/tree/main/app/android/app/src/main): Kotlin bridge · Android native activity, models, and 3D rendering assets. · `app/android/app/src/main`

### Robot firmware

ESP32-S3 firmware that owns the physical robot.

- [Main app](https://github.com/m5stack/stackchan/blob/main/firmware/main/main.cpp): embedded entry · ESP-IDF entrypoint and runtime wiring for the robot firmware. · `firmware/main/main.cpp`
- [HAL layer](https://github.com/m5stack/stackchan/tree/main/firmware/main/hal): hardware abstraction · Board-specific hardware access for audio, BLE, ESP-NOW, network, OTA, RTC, servos, IMU, camera, display, touch, and glue. · `firmware/main/hal`
- [Behavior engine](https://github.com/m5stack/stackchan/tree/main/firmware/main/stackchan): stackchan domain · Avatar, modifiers, motion, animation, JSON helpers, and utility math for robot behavior. · `firmware/main/stackchan`
- [Robot apps](https://github.com/m5stack/stackchan/tree/main/firmware/main/apps): mode framework · User-facing firmware modes such as launcher, setup, app center, AI agent, avatar, dance, and EZ data. · `firmware/main/apps`
- [Firmware assets](https://github.com/m5stack/stackchan/tree/main/firmware/main/assets): embedded media · Icons, fonts, sounds, and setup imagery bundled into the robot runtime. · `firmware/main/assets`

### Remote controller

ESP32 handheld controller with local ESP-NOW input.

- [Remote app](https://github.com/m5stack/stackchan/blob/main/remote/code/main/StackChan-RemoteControl-ESPNow.cpp): ESP-IDF entry · Main firmware for the handheld remote controller. · `remote/code/main/StackChan-RemoteControl-ESPNow.cpp`
- [ESP-NOW link](https://github.com/m5stack/stackchan/tree/main/remote/code/main/esp_now): radio transport · Low-latency peer-to-peer control initialization and communication. · `remote/code/main/esp_now`
- [Remote UI](https://github.com/m5stack/stackchan/tree/main/remote/code/main/ui): LVGL screens · Setup, running, and IMU views for the standalone controller. · `remote/code/main/ui`
- [Joystick input](https://github.com/m5stack/stackchan/tree/main/remote/code/main/joystick): controls · Joystick handling and input mapping for motion control. · `remote/code/main/joystick`

### Backend server

Go control plane for accounts, content, and device management.

- [API surface](https://github.com/m5stack/stackchan/tree/main/server/api): versioned endpoints · Admin, app store, dance, device, file, friend, pano, post, stackchan device, user, and xiaozhi APIs. · `server/api`
- [Controllers](https://github.com/m5stack/stackchan/tree/main/server/internal/controller): HTTP handlers · Maps APIs to application services and request workflows. · `server/internal/controller`
- [Services](https://github.com/m5stack/stackchan/tree/main/server/internal/service): business logic · Domain logic for users, devices, apps, dances, files, and agents. · `server/internal/service`
- [Data layer](https://github.com/m5stack/stackchan/tree/main/server/internal/dao): DAO/model · Persistence models and DAOs backed by relational storage and generated internal entities. · `server/internal/dao`
- [WebSocket hub](https://github.com/m5stack/stackchan/tree/main/server/internal/web_socket): live sessions · Socket support for live device or avatar session updates. · `server/internal/web_socket`
- [Deploy manifests](https://github.com/m5stack/stackchan/tree/main/server/manifest): container/kustomize · Docker and Kustomize deployment configuration. · `server/manifest`

### Shared assets

Common media and 3D resources used across runtimes.

- [Shared media](https://github.com/m5stack/stackchan/tree/main/app/assets): cross-platform assets · 3D models, setup videos, music, icons, and UI artwork used across app and firmware. · `app/assets`

## Connections

- Flutter app → App transport: uses · Flutter UI uses transport utilities for backend and device communication.
- Flutter app → App models: binds · Screens and flows bind to product domain models.
- Flutter app → iOS native: bridges · iOS-specific rendering and motion views are launched from the app.
- Flutter app → Android native: bridges · Android native layer supports local robot visualization.
- App transport → API surface: HTTP/WebSocket · App talks to backend endpoints and live sessions.
- Flutter app → HAL layer: device control · App coordinates setup, binding, monitoring, and control with the robot.
- Main app → HAL layer: initializes · Entrypoint wires board services and hardware subsystems.
- Main app → Behavior engine: runs · Main runtime drives stackchan behavior state and actuation.
- Main app → Robot apps: hosts · Firmware exposes multiple user-facing modes and mini-apps.
- Robot apps → Behavior engine: invokes · App modes reuse avatar, motion, and modifier primitives.
- Behavior engine → HAL layer: acts on · Behavior layer ultimately drives servos, display, audio, camera, and sensors through HAL.
- HAL layer → Firmware assets: loads · Robot runtime consumes embedded icons, sounds, and setup art.
- Remote app → Joystick input: reads · Remote firmware polls joystick and local controls.
- Remote app → ESP-NOW link: sends · Main remote firmware communicates over ESP-NOW.
- Remote app → Remote UI: renders · Local UI shows setup, running, and IMU state.
- ESP-NOW link → HAL layer: peer control · Remote controller directly drives the robot via low-latency radio link.
- API surface → Controllers: dispatches · Versioned APIs are implemented by controllers.
- Controllers → Services: calls · Controllers delegate business logic to services.
- Services → Data layer: persists · Services read and write through DAOs and generated models.
- Controllers → WebSocket hub: upgrades · Some workflows use WebSocket sessions for live updates.
- API surface → Deploy manifests: deploys · Backend ships with container and Kustomize manifests.
- Shared media → Flutter app: consumed by · Flutter UI uses common art, video, and model assets.
- Shared media → iOS native: consumed by · iOS native views render shared 3D assets.
- Shared media → Android native: consumed by · Android native layer includes shared 3D model assets.
- Shared media → Firmware assets: mirrored in · Embedded runtime bundles a subset of shared content.
- Services → Main app: orchestrates · Backend supports registration, binding, content, and token workflows for devices.
