How facebookincubator/muse-gadget-sdk works
Muse Gadgets provides two device SDKs: ESP32 firmware and a Linux/Raspberry Pi service. The principal workflow is pairing a gadget with the Muse app using an SDK token, establishing a device-to-VM connection, then presenting Muse interactions through device hardware or executing Linux commands. The ESP32 SDK also covers UI, connectivity, voice, and hardware-specific integrations; Linux adds local service control and command execution. Several implementation links are documented or visible in the source index, but unsampled firmware details are not inferred.
Main entry points: service.py Device service · muse_glue.c Muse integration · app.c App runtime · muse_ui.c UI and display
Components 29
ESP32 firmware
- Firmware entry
esp32/main/main.c - App runtime
esp32/main/app.c - Pairing and BLE
esp32/main/link_pairing.c - Wi-Fi and discovery
esp32/main/wifi_mgr.c - Secure tunnel
esp32/main/noise_tunnel.cpp - Firmware updates
esp32/main/ota.c - Diagnostic logging
esp32/main/diagnostic_log.h - Status indicators
esp32/main/led_status.c
Muse device runtime
- Muse integration
esp32/main/muse_glue.c - UI and display
esp32/components/muse/muse_ui.c - Pixel renderer
esp32/components/muse/muse_pixel.h - Settings and state
esp32/components/muse/muse_settings.c - Voice chat
esp32/components/muse/muse_chat_session.cpp - Audio playback
esp32/components/minimp3/include/minimp3.h - Board support
esp32/components/muse/boards
Linux SDK
- Device service
linux/src/musegadget/service.py - Muse API
linux/src/musegadget/muse_api.py - Link client
linux/src/musegadget/link_client.py - Command executor
linux/src/musegadget/executor.py - Pairing
linux/src/musegadget/pairing.py
Secure connectivity
- Noise transport
linux/src/musegadget/noise/transport.py - ESP32 Noise core
esp32/components/noise_core/src/ClientSession.cpp
Desktop preview
- UI simulator
esp32/simulator/src/main.c - Simulated services
esp32/simulator/src/sim_services.c
Other components
- Gadget builder
- Muse app
- Muse services
- Device hardware
- Local commands
Connections 30
- Gadget builder → Device hardware: builds device
README.md - Muse app → Pairing and BLE: pairs gadget
README.md - Gadget builder → Device service: sets up SDK
README.md - Firmware entry → Muse integration: starts
esp32/main/main.c - Firmware entry → App runtime: runs
esp32/main/main.c - App runtime → Pairing and BLE: manages setup
- App runtime → Wi-Fi and discovery: manages connectivity
- App runtime → Secure tunnel: establishes session
- App runtime → Firmware updates: handles updates
- Muse integration → Device hardware: uses hardware
- Muse integration → UI and display: integrates UI
- UI and display → Pixel renderer: renders
- UI and display → Settings and state: reads settings
- Muse integration → Voice chat: integrates chat
- Voice chat → Audio playback: uses decoder
esp32/components/muse/muse_chat_session.cpp - Muse integration → Board support: selects board
- Device service → Muse API: refreshes credentials
linux/src/musegadget/service.py - Device service → Link client: opens session
linux/src/musegadget/service.py - Device service → Command executor: runs commands
linux/src/musegadget/service.py - Device service → Pairing: uses pairing
- Device service → Local commands: exposes commands
- Link client → Noise transport: secures transport
linux/src/musegadget/link_client.py - Link client → Muse API: imports API
linux/src/musegadget/link_client.py - Device service → Muse services: connects to VM
README.md - Secure tunnel → Muse services: connects securely
- ESP32 Noise core → Muse services: secures session
- UI simulator → Simulated services: uses simulated state
esp32/simulator/src/main.c - UI simulator → UI and display: previews UI
- Firmware entry → Diagnostic logging: initializes logging
esp32/main/main.c - Muse integration → Status indicators: integrates indicators