# facebookincubator/muse-gadget-sdk architecture

> Architecture diagram of the GitHub repository facebookincubator/muse-gadget-sdk, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-10-07). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/facebookincubator/muse-gadget-sdk
- Repository: https://github.com/facebookincubator/muse-gadget-sdk

## Overview

Muse Gadgets provides two device SDKs: ESP32 firmware and a Linux/Raspberry Pi service. The principal workflow is pairing a gadget with the Muse app using an SDK token, establishing a device-to-VM connection, then presenting Muse interactions through device hardware or executing Linux commands. The ESP32 SDK also covers UI, connectivity, voice, and hardware-specific integrations; Linux adds local service control and command execution. Several implementation links are documented or visible in the source index, but unsampled firmware details are not inferred.

Main entry points: [`linux/src/musegadget/service.py`](https://github.com/facebookincubator/muse-gadget-sdk/blob/main/linux/src/musegadget/service.py) (Device service), [`esp32/main/muse_glue.c`](https://github.com/facebookincubator/muse-gadget-sdk/blob/main/esp32/main/muse_glue.c) (Muse integration), [`esp32/main/app.c`](https://github.com/facebookincubator/muse-gadget-sdk/blob/main/esp32/main/app.c) (App runtime), [`esp32/components/muse/muse_ui.c`](https://github.com/facebookincubator/muse-gadget-sdk/blob/main/esp32/components/muse/muse_ui.c) (UI and display)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_esp32["ESP32 firmware"]
  node_esp_entry["Firmware entry<br/>[main.c]"]
  node_esp_app["App runtime<br/>[app.c]"]
  node_pairing["Pairing and BLE<br/>[link_pairing.c]"]
  node_network["Wi-Fi and discovery<br/>[wifi_mgr.c]"]
  node_tunnel["Secure tunnel<br/>[noise_tunnel.cpp]"]
  node_ota["Firmware updates<br/>[ota.c]"]
  node_diagnostics["Diagnostic logging<br/>[diagnostic_log.h]"]
  node_led["Status indicators<br/>[led_status.c]"]
end

subgraph group_muse["Muse device runtime"]
  node_muse_glue["Muse integration<br/>[muse_glue.c]"]
  node_muse_ui["UI and display<br/>[muse_ui.c]"]
  node_pixel["Pixel renderer<br/>[muse_pixel.h]"]
  node_settings["Settings and state<br/>[muse_settings.c]"]
  node_chat["Voice chat"]
  node_audio["Audio playback<br/>[minimp3.h]"]
  node_boards["Board support"]
end

subgraph group_linux["Linux SDK"]
  node_linux_service["Device service<br/>[service.py]"]
  node_linux_api["Muse API<br/>[muse_api.py]"]
  node_link_client["Link client<br/>[link_client.py]"]
  node_executor["Command executor<br/>[executor.py]"]
  node_linux_pair["Pairing<br/>[pairing.py]"]
end

subgraph group_secure["Secure connectivity"]
  node_noise["Noise transport<br/>[transport.py]"]
  node_noise_core["ESP32 Noise core<br/>[ClientSession.cpp]"]
end

subgraph group_preview["Desktop preview"]
  node_simulator["UI simulator<br/>[main.c]"]
  node_sim_services["Simulated services<br/>[sim_services.c]"]
end

node_user(("Gadget builder"))
node_muse_app(("Muse app"))
node_muse_service{{"Muse services"}}
node_device_hw{{"Device hardware"}}
node_linux_cmds{{"Local commands"}}

node_user -->|"builds device"| node_device_hw
node_muse_app -->|"pairs gadget"| node_pairing
node_user -->|"sets up SDK"| node_linux_service
node_esp_entry -->|"starts"| node_muse_glue
node_esp_entry -->|"runs"| node_esp_app
node_esp_app -.->|"manages setup"| node_pairing
node_esp_app -.->|"manages connectivity"| node_network
node_esp_app -.->|"establishes session"| node_tunnel
node_esp_app -.->|"handles updates"| node_ota
node_muse_glue -.->|"uses hardware"| node_device_hw
node_muse_glue -.->|"integrates UI"| node_muse_ui
node_muse_ui -.->|"renders"| node_pixel
node_muse_ui -.->|"reads settings"| node_settings
node_muse_glue -.->|"integrates chat"| node_chat
node_chat -->|"uses decoder"| node_audio
node_muse_glue -.->|"selects board"| node_boards
node_linux_service -->|"refreshes credentials"| node_linux_api
node_linux_service -->|"opens session"| node_link_client
node_linux_service -->|"runs commands"| node_executor
node_linux_service -.->|"uses pairing"| node_linux_pair
node_linux_service -.->|"exposes commands"| node_linux_cmds
node_link_client -->|"secures transport"| node_noise
node_link_client -->|"imports API"| node_linux_api
node_linux_service -->|"connects to VM"| node_muse_service
node_tunnel -.->|"connects securely"| node_muse_service
node_noise_core -.->|"secures session"| node_muse_service
node_simulator -->|"uses simulated state"| node_sim_services
node_simulator -.->|"previews UI"| node_muse_ui
node_esp_entry -->|"initializes logging"| node_diagnostics
node_muse_glue -->|"integrates indicators"| node_led

click node_esp_entry "https://github.com/facebookincubator/muse-gadget-sdk/blob/main/esp32/main/main.c"
click node_esp_app "https://github.com/facebookincubator/muse-gadget-sdk/blob/main/esp32/main/app.c"
click node_pairing "https://github.com/facebookincubator/muse-gadget-sdk/blob/main/esp32/main/link_pairing.c"
click node_network "https://github.com/facebookincubator/muse-gadget-sdk/blob/main/esp32/main/wifi_mgr.c"
click node_tunnel "https://github.com/facebookincubator/muse-gadget-sdk/blob/main/esp32/main/noise_tunnel.cpp"
click node_ota "https://github.com/facebookincubator/muse-gadget-sdk/blob/main/esp32/main/ota.c"
click node_muse_glue "https://github.com/facebookincubator/muse-gadget-sdk/blob/main/esp32/main/muse_glue.c"
click node_muse_ui "https://github.com/facebookincubator/muse-gadget-sdk/blob/main/esp32/components/muse/muse_ui.c"
click node_pixel "https://github.com/facebookincubator/muse-gadget-sdk/blob/main/esp32/components/muse/muse_pixel.h"
click node_settings "https://github.com/facebookincubator/muse-gadget-sdk/blob/main/esp32/components/muse/muse_settings.c"
click node_chat "https://github.com/facebookincubator/muse-gadget-sdk/blob/main/esp32/components/muse/muse_chat_session.cpp"
click node_audio "https://github.com/facebookincubator/muse-gadget-sdk/blob/main/esp32/components/minimp3/include/minimp3.h"
click node_boards "https://github.com/facebookincubator/muse-gadget-sdk/tree/main/esp32/components/muse/boards"
click node_linux_service "https://github.com/facebookincubator/muse-gadget-sdk/blob/main/linux/src/musegadget/service.py"
click node_linux_api "https://github.com/facebookincubator/muse-gadget-sdk/blob/main/linux/src/musegadget/muse_api.py"
click node_link_client "https://github.com/facebookincubator/muse-gadget-sdk/blob/main/linux/src/musegadget/link_client.py"
click node_executor "https://github.com/facebookincubator/muse-gadget-sdk/blob/main/linux/src/musegadget/executor.py"
click node_linux_pair "https://github.com/facebookincubator/muse-gadget-sdk/blob/main/linux/src/musegadget/pairing.py"
click node_noise "https://github.com/facebookincubator/muse-gadget-sdk/blob/main/linux/src/musegadget/noise/transport.py"
click node_noise_core "https://github.com/facebookincubator/muse-gadget-sdk/blob/main/esp32/components/noise_core/src/ClientSession.cpp"
click node_simulator "https://github.com/facebookincubator/muse-gadget-sdk/blob/main/esp32/simulator/src/main.c"
click node_sim_services "https://github.com/facebookincubator/muse-gadget-sdk/blob/main/esp32/simulator/src/sim_services.c"
click node_diagnostics "https://github.com/facebookincubator/muse-gadget-sdk/blob/main/esp32/main/diagnostic_log.h"
click node_led "https://github.com/facebookincubator/muse-gadget-sdk/blob/main/esp32/main/led_status.c"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_esp_entry,node_esp_app,node_pairing,node_network,node_tunnel,node_ota,node_diagnostics,node_led toneBlue
class node_muse_glue,node_muse_ui,node_pixel,node_settings,node_chat,node_audio,node_boards toneAmber
class node_linux_service,node_linux_api,node_link_client,node_executor,node_linux_pair toneMint
class node_noise,node_noise_core toneRose
class node_simulator,node_sim_services,node_user,node_muse_app,node_muse_service,node_device_hw,node_linux_cmds toneIndigo
```

## Components

### ESP32 firmware

- [Firmware entry](https://github.com/facebookincubator/muse-gadget-sdk/blob/main/esp32/main/main.c): `esp32/main/main.c`
- [App runtime](https://github.com/facebookincubator/muse-gadget-sdk/blob/main/esp32/main/app.c): `esp32/main/app.c`
- [Pairing and BLE](https://github.com/facebookincubator/muse-gadget-sdk/blob/main/esp32/main/link_pairing.c): `esp32/main/link_pairing.c`
- [Wi-Fi and discovery](https://github.com/facebookincubator/muse-gadget-sdk/blob/main/esp32/main/wifi_mgr.c): `esp32/main/wifi_mgr.c`
- [Secure tunnel](https://github.com/facebookincubator/muse-gadget-sdk/blob/main/esp32/main/noise_tunnel.cpp): `esp32/main/noise_tunnel.cpp`
- [Firmware updates](https://github.com/facebookincubator/muse-gadget-sdk/blob/main/esp32/main/ota.c): `esp32/main/ota.c`
- [Diagnostic logging](https://github.com/facebookincubator/muse-gadget-sdk/blob/main/esp32/main/diagnostic_log.h): `esp32/main/diagnostic_log.h`
- [Status indicators](https://github.com/facebookincubator/muse-gadget-sdk/blob/main/esp32/main/led_status.c): `esp32/main/led_status.c`

### Muse device runtime

- [Muse integration](https://github.com/facebookincubator/muse-gadget-sdk/blob/main/esp32/main/muse_glue.c): `esp32/main/muse_glue.c`
- [UI and display](https://github.com/facebookincubator/muse-gadget-sdk/blob/main/esp32/components/muse/muse_ui.c): `esp32/components/muse/muse_ui.c`
- [Pixel renderer](https://github.com/facebookincubator/muse-gadget-sdk/blob/main/esp32/components/muse/muse_pixel.h): `esp32/components/muse/muse_pixel.h`
- [Settings and state](https://github.com/facebookincubator/muse-gadget-sdk/blob/main/esp32/components/muse/muse_settings.c): `esp32/components/muse/muse_settings.c`
- [Voice chat](https://github.com/facebookincubator/muse-gadget-sdk/blob/main/esp32/components/muse/muse_chat_session.cpp): `esp32/components/muse/muse_chat_session.cpp`
- [Audio playback](https://github.com/facebookincubator/muse-gadget-sdk/blob/main/esp32/components/minimp3/include/minimp3.h): `esp32/components/minimp3/include/minimp3.h`
- [Board support](https://github.com/facebookincubator/muse-gadget-sdk/tree/main/esp32/components/muse/boards): `esp32/components/muse/boards`

### Linux SDK

- [Device service](https://github.com/facebookincubator/muse-gadget-sdk/blob/main/linux/src/musegadget/service.py): `linux/src/musegadget/service.py`
- [Muse API](https://github.com/facebookincubator/muse-gadget-sdk/blob/main/linux/src/musegadget/muse_api.py): `linux/src/musegadget/muse_api.py`
- [Link client](https://github.com/facebookincubator/muse-gadget-sdk/blob/main/linux/src/musegadget/link_client.py): `linux/src/musegadget/link_client.py`
- [Command executor](https://github.com/facebookincubator/muse-gadget-sdk/blob/main/linux/src/musegadget/executor.py): `linux/src/musegadget/executor.py`
- [Pairing](https://github.com/facebookincubator/muse-gadget-sdk/blob/main/linux/src/musegadget/pairing.py): `linux/src/musegadget/pairing.py`

### Secure connectivity

- [Noise transport](https://github.com/facebookincubator/muse-gadget-sdk/blob/main/linux/src/musegadget/noise/transport.py): `linux/src/musegadget/noise/transport.py`
- [ESP32 Noise core](https://github.com/facebookincubator/muse-gadget-sdk/blob/main/esp32/components/noise_core/src/ClientSession.cpp): `esp32/components/noise_core/src/ClientSession.cpp`

### Desktop preview

- [UI simulator](https://github.com/facebookincubator/muse-gadget-sdk/blob/main/esp32/simulator/src/main.c): `esp32/simulator/src/main.c`
- [Simulated services](https://github.com/facebookincubator/muse-gadget-sdk/blob/main/esp32/simulator/src/sim_services.c): `esp32/simulator/src/sim_services.c`

### Other components

- **Gadget builder**
- **Muse app**
- **Muse services**
- **Device hardware**
- **Local commands**

## Connections

- Gadget builder → Device hardware: builds device (evidence: `README.md`)
- Muse app → Pairing and BLE: pairs gadget (evidence: `README.md`)
- Gadget builder → Device service: sets up SDK (evidence: `README.md`)
- Firmware entry → Muse integration: starts (evidence: `esp32/main/main.c`)
- Firmware entry → App runtime: runs (evidence: `esp32/main/main.c`)
- App runtime → Pairing and BLE: manages setup
- App runtime → Wi-Fi and discovery: manages connectivity
- App runtime → Secure tunnel: establishes session
- App runtime → Firmware updates: handles updates
- Muse integration → Device hardware: uses hardware
- Muse integration → UI and display: integrates UI
- UI and display → Pixel renderer: renders
- UI and display → Settings and state: reads settings
- Muse integration → Voice chat: integrates chat
- Voice chat → Audio playback: uses decoder (evidence: `esp32/components/muse/muse_chat_session.cpp`)
- Muse integration → Board support: selects board
- Device service → Muse API: refreshes credentials (evidence: `linux/src/musegadget/service.py`)
- Device service → Link client: opens session (evidence: `linux/src/musegadget/service.py`)
- Device service → Command executor: runs commands (evidence: `linux/src/musegadget/service.py`)
- Device service → Pairing: uses pairing
- Device service → Local commands: exposes commands
- Link client → Noise transport: secures transport (evidence: `linux/src/musegadget/link_client.py`)
- Link client → Muse API: imports API (evidence: `linux/src/musegadget/link_client.py`)
- Device service → Muse services: connects to VM (evidence: `README.md`)
- Secure tunnel → Muse services: connects securely
- ESP32 Noise core → Muse services: secures session
- UI simulator → Simulated services: uses simulated state (evidence: `esp32/simulator/src/main.c`)
- UI simulator → UI and display: previews UI
- Firmware entry → Diagnostic logging: initializes logging (evidence: `esp32/main/main.c`)
- Muse integration → Status indicators: integrates indicators
