# livekit/client-sdk-esp32 architecture

> Architecture diagram of the GitHub repository livekit/client-sdk-esp32, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-10-25). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/livekit/client-sdk-esp32
- Repository: https://github.com/livekit/client-sdk-esp32

## Overview

##### Main Components
- **Application Layer (Examples)**
- minimal, minimal_video, voice_agent, etc.
- Uses `livekit_system_init()`, configures capturer & renderer, creates/connects `livekit_room`.
- **Example Utilities** (`components/example_utils`)
- Helper functions for networking, logging, JSON, token handling.
- **LiveKit Core Component** (`components/livekit`)
Subcomponents:
- **System** – initialization, event loop.
- **Engine** – media pipeline orchestration.
- **Signaling** – WebSocket/HTTP handshake with LiveKit server.
- **Protocol** – Encode/decode protobuf messages.
- **RPC Manager** – remote method calls.
- **Peer Manager** – tracks participants & their media streams.
- **Protocol Definitions** (`components/livekit/protocol`)
- `.proto` files → generated C via Nanopb.
- **Token Management** (`components/sandbox_token`)
- Generates or manages JWT tokens for auth (in sandbox mode).
- **Third‐Party Libraries** (`components/third_party`)
- **Nanopb** – lightweight Protobuf C implementation.
- **Khash** – hash‐map utility for data structures.
- **Hardware Drivers & IDF Components**
- **esp_capture** – camera/mic capture (I2S, MIPI CSI).
- **av_render** – audio/video output (I2S, LCD).
- Wi-Fi / TCP/IP stack of ESP-IDF.
- **External Service**
- **LiveKit Server** (cloud or self‐hosted) – signaling & SFU for media routing.

##### Relationships & Data Flows
- Application ⇄ Example Utils: calls helper APIs (e.g., JSON, token creation).
- Application → LiveKit Core:
- `livekit_system_init()`, `livekit_room_create()`, `livekit_room_connect()`.
- LiveKit Core → Signaling → TCP/IP → LiveKit Server
- LiveKit Core → Protocol (Nanopb) → encode/decode messages
- LiveKit Core → RPC Manager → invoke handlers or send RPCs
- LiveKit Core ↔ Engine → manage media tracks
- Engine → esp_capture (audio/video capture) → media frames
- Engine → codecs (H.264, Opus) → encoded frames
- Engine → TCP/IP → LiveKit Server
- LiveKit Server → TCP/IP → Engine → av_render → hardware outputs
- Sandbox Token component: optional path App → sandbox_token → LiveKit Server auth

##### Architectural Patterns & Principles
- **Modular IDF Components**: each feature in its own CMake/`Kconfig.projbuild` component.
- **Layered Design**:
- Hardware Drivers → Media Engine → Protocol/Signaling → Application API.
- **Event-Driven**: callbacks (on_state_changed, on_participant_info, RPC handlers).
- **Plugin Hooks**: user provides capturer/renderer handles.

##### Relevant Technologies, Frameworks & Libraries
- ESP-IDF build system (CMake, Kconfig)
- Nanopb (protobuf C)
- Khash
- Opus & H.264 codecs via ESP-IDF components
- WebSocket/HTTP stack of ESP-IDF
- esp_capture, av_render
- C standard library, lwIP

##### Elements
- Media data flow (camera → Engine → network → SFU → Engine → display)
- Control/signaling flow (App API → Signaling → Server)
- RPC calls (Engine ↔ RPC Manager ↔ App Handlers).
- Blue for IDF Components (esp_capture, av_render)
- Green for LiveKit Core modules
- Orange for Third‐Party libraries
- Yellow for Application layer
- Gray for External services (LiveKit Server).

## Diagram (Mermaid)

```mermaid
flowchart TB
    LiveKitServer["LiveKit Server"]:::external
    LiveKitServer -->|Protobuf over TCP\nSignaling| wifi
    LiveKitServer -->|Media over TCP| wifi

    subgraph "Application Layer"
        minimal["minimal"]:::app
        minimal_video["minimal_video"]:::app
        voice_agent["voice_agent"]:::app
        example_utils["Example Utils"]:::app
        sandbox_token["Sandbox Token"]:::app
    end

    minimal --> example_utils
    minimal_video --> example_utils
    voice_agent --> example_utils

    minimal --> entry
    minimal_video --> entry
    voice_agent --> entry

    minimal --> sandbox_token
    minimal_video --> sandbox_token
    voice_agent --> sandbox_token
    sandbox_token --> entry

    subgraph "LiveKit Core"
        entry["Entry Point\n(livekit.c)"]:::core
        system["System"]:::core
        engine["Engine"]:::core
        signaling["Signaling"]:::core
        protocolCore["Protocol"]:::core
        rpcManager["RPC Manager"]:::core
        peerManager["Peer Manager"]:::core
        utilsCore["URL & Utils"]:::core
    end

    entry --> system
    system --> engine
    entry --> signaling
    signaling --> protocolCore
    signaling --> rpcManager
    signaling --> peerManager

    protocolCore --> nanopb
    rpcManager --> utilsCore
    peerManager --> utilsCore

    engine --> wifi
    signaling --> wifi

    subgraph "Third-Party Libraries"
        nanopb["Nanopb"]:::third
        khash["Khash"]:::third
    end

    utilsCore --> khash

    wifi --> device

    subgraph "ESP32 Chip"
        device["ESP32 Chip"]:::device
    end

    device --> esp_capture
    device --> av_render
    device --> wifi

    subgraph "Hardware Drivers & IDF Components"
        esp_capture["esp_capture"]:::driver
        av_render["av_render"]:::driver
        wifi["WiFi/TCP-IP"]:::driver
    end

    click minimal "https://github.com/livekit/client-sdk-esp32/tree/main/components/livekit/examples/minimal"
    click minimal_video "https://github.com/livekit/client-sdk-esp32/tree/main/components/livekit/examples/minimal_video"
    click voice_agent "https://github.com/livekit/client-sdk-esp32/tree/main/components/livekit/examples/voice_agent"
    click example_utils "https://github.com/livekit/client-sdk-esp32/tree/main/components/example_utils"
    click sandbox_token "https://github.com/livekit/client-sdk-esp32/tree/main/components/sandbox_token"
    click entry "https://github.com/livekit/client-sdk-esp32/blob/main/components/livekit/core/livekit.c"
    click system "https://github.com/livekit/client-sdk-esp32/blob/main/components/livekit/core/system.c"
    click engine "https://github.com/livekit/client-sdk-esp32/blob/main/components/livekit/core/engine.c"
    click signaling "https://github.com/livekit/client-sdk-esp32/blob/main/components/livekit/core/signaling.c"
    click protocolCore "https://github.com/livekit/client-sdk-esp32/blob/main/components/livekit/core/protocol.c"
    click rpcManager "https://github.com/livekit/client-sdk-esp32/blob/main/components/livekit/core/rpc_manager.c"
    click peerManager "https://github.com/livekit/client-sdk-esp32/blob/main/components/livekit/core/peer.c"
    click utilsCore "https://github.com/livekit/client-sdk-esp32/blob/main/components/livekit/core/utils.c"
    click nanopb "https://github.com/livekit/client-sdk-esp32/tree/main/components/third_party/nanopb"
    click khash "https://github.com/livekit/client-sdk-esp32/tree/main/components/third_party/khash"

    classDef app fill:#fff2cc,stroke:#f1c232,color:#000
    classDef core fill:#d9ead3,stroke:#93c47d,color:#000
    classDef third fill:#fce5cd,stroke:#f4b183,color:#000
    classDef driver fill:#cfe2f3,stroke:#6fa8dc,color:#000
    classDef external fill:#e0e0e0,stroke:#a6a6a6,color:#000
    classDef device fill:#d9d9d9,stroke:#858585,color:#000
```
