# kun-acm/esp32s3-av-monitor architecture

> Architecture diagram of the GitHub repository kun-acm/esp32s3-av-monitor, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-09-30). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/kun-acm/esp32s3-av-monitor
- Repository: https://github.com/kun-acm/esp32s3-av-monitor

## Overview

This is ESP32-S3 firmware for LAN audio/video monitoring, local face detection, and an LCD preview. The principal workflow captures OV5640 JPEG frames into a latest-frame cache, serves MJPEG to a browser, and separately decodes frames for face detection and LCD display. Audio capture/playback uses a distinct WebSocket path; HTTP also exposes status, snapshots, face results, and light controls. Relationships follow the README’s documented data flow and the sampled startup and handler code; several subsystem implementations were not excerpted, so uncertain internal calls are omitted.

Main entry points: [`main/APP/av_stream.c`](https://github.com/kun-acm/esp32s3-av-monitor/blob/main/main/APP/av_stream.c) (HTTP and MJPEG), [`main/APP/camera_preview.c`](https://github.com/kun-acm/esp32s3-av-monitor/blob/main/main/APP/camera_preview.c) (Preview Decoder), [`main/APP/audio_tcp.c`](https://github.com/kun-acm/esp32s3-av-monitor/blob/main/main/APP/audio_tcp.c) (Audio WebSocket), [`main/APP/camera_frames.c`](https://github.com/kun-acm/esp32s3-av-monitor/blob/main/main/APP/camera_frames.c) (JPEG Frame Cache)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_capture["Camera Pipeline"]
  node_camera_init["Camera Setup<br/>[camera_init.c]"]
  node_frames[("JPEG Frame Cache<br/>[camera_frames.c]")]
  node_jpeg_decoder["JPEG Decoder API<br/>[esp_jpeg_dec.h]"]
  node_preview["Preview Decoder<br/>[camera_preview.c]"]
end

subgraph group_services["Network Services"]
  node_wifi["Wi-Fi STA<br/>[wifi_sta.c]"]
  node_av_http["HTTP and MJPEG<br/>[av_stream.c]"]
  node_audio_ws["Audio WebSocket<br/>[audio_tcp.c]"]
end

subgraph group_analysis["Detection and Display"]
  node_face_task["Face Detection Task<br/>[face_detect_task.c]"]
  node_face_model["MSR01 Detector<br/>[face_detect.cpp]"]
  node_lcd_video["LCD Video Renderer<br/>[lcd_video.c]"]
  node_lcd_driver["ILI9341 Driver<br/>[lcd_drv.c]"]
  node_lcd_status["LCD Status Overlay<br/>[lcd_display.c]"]
  node_motion["Motion Detection<br/>[motion_detect.c]"]
end

subgraph group_device["Audio and Controls"]
  node_audio_capture["I2S Audio Capture<br/>[audio_capture.c]"]
  node_audio_playback["I2S Audio Playback<br/>[audio_playback.c]"]
  node_mulaw["μ-law Codec<br/>[mulaw.c]"]
  node_flash["Light Controller<br/>[flash_light.c]"]
  node_volume["Volume and Keys<br/>[volume_ctrl.c]"]
end

node_operator(("LAN Browser"))
node_camera["OV5640 Camera"]

node_operator -->|"connects over LAN"| node_wifi
node_camera_init -->|"configures"| node_camera
node_camera -->|"captures JPEG"| node_frames
node_frames -->|"copies frames"| node_preview
node_preview -->|"decodes with"| node_jpeg_decoder
node_frames -->|"serves frame copies"| node_av_http
node_av_http -->|"streams MJPEG and HTTP"| node_operator
node_preview -->|"provides RGB preview"| node_face_task
node_face_task -->|"runs inference"| node_face_model
node_face_task -->|"publishes face results"| node_av_http
node_preview -->|"provides RGB preview"| node_lcd_video
node_lcd_video -->|"renders over SPI"| node_lcd_driver
node_lcd_status -->|"draws status"| node_lcd_driver
node_audio_capture -->|"supplies captured audio"| node_audio_ws
node_audio_ws -->|"encodes and decodes"| node_mulaw
node_audio_ws -->|"queues received audio"| node_audio_playback
node_audio_ws -->|"exchanges audio"| node_operator
node_av_http -->|"handles light controls"| node_flash
node_volume -->|"controls audio level"| node_audio_capture
node_operator -->|"requests controls and results"| node_av_http
node_motion -.-> node_av_http

click node_wifi "https://github.com/kun-acm/esp32s3-av-monitor/blob/main/main/APP/wifi_sta.c"
click node_camera_init "https://github.com/kun-acm/esp32s3-av-monitor/blob/main/main/APP/camera_init.c"
click node_frames "https://github.com/kun-acm/esp32s3-av-monitor/blob/main/main/APP/camera_frames.c"
click node_jpeg_decoder "https://github.com/kun-acm/esp32s3-av-monitor/blob/main/components/esp_new_jpeg/include/esp_jpeg_dec.h"
click node_preview "https://github.com/kun-acm/esp32s3-av-monitor/blob/main/main/APP/camera_preview.c"
click node_av_http "https://github.com/kun-acm/esp32s3-av-monitor/blob/main/main/APP/av_stream.c"
click node_face_task "https://github.com/kun-acm/esp32s3-av-monitor/blob/main/main/APP/face_detect_task.c"
click node_face_model "https://github.com/kun-acm/esp32s3-av-monitor/blob/main/main/APP/face_detect.cpp"
click node_lcd_video "https://github.com/kun-acm/esp32s3-av-monitor/blob/main/main/APP/lcd_video.c"
click node_lcd_driver "https://github.com/kun-acm/esp32s3-av-monitor/blob/main/main/APP/lcd_drv.c"
click node_lcd_status "https://github.com/kun-acm/esp32s3-av-monitor/blob/main/main/APP/lcd_display.c"
click node_audio_capture "https://github.com/kun-acm/esp32s3-av-monitor/blob/main/main/APP/audio_capture.c"
click node_audio_playback "https://github.com/kun-acm/esp32s3-av-monitor/blob/main/main/APP/audio_playback.c"
click node_mulaw "https://github.com/kun-acm/esp32s3-av-monitor/blob/main/main/APP/mulaw.c"
click node_audio_ws "https://github.com/kun-acm/esp32s3-av-monitor/blob/main/main/APP/audio_tcp.c"
click node_flash "https://github.com/kun-acm/esp32s3-av-monitor/blob/main/main/APP/flash_light.c"
click node_volume "https://github.com/kun-acm/esp32s3-av-monitor/blob/main/main/APP/volume_ctrl.c"
click node_motion "https://github.com/kun-acm/esp32s3-av-monitor/blob/main/main/APP/motion_detect.c"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_camera_init,node_frames,node_jpeg_decoder,node_preview,node_operator toneBlue
class node_wifi,node_av_http,node_audio_ws toneAmber
class node_face_task,node_face_model,node_lcd_video,node_lcd_driver,node_lcd_status,node_motion toneMint
class node_audio_capture,node_audio_playback,node_mulaw,node_flash,node_volume toneRose
class node_camera toneIndigo
```

## Components

### Camera Pipeline

- [Camera Setup](https://github.com/kun-acm/esp32s3-av-monitor/blob/main/main/APP/camera_init.c): `main/APP/camera_init.c`
- [JPEG Frame Cache](https://github.com/kun-acm/esp32s3-av-monitor/blob/main/main/APP/camera_frames.c): `main/APP/camera_frames.c`
- [JPEG Decoder API](https://github.com/kun-acm/esp32s3-av-monitor/blob/main/components/esp_new_jpeg/include/esp_jpeg_dec.h): `components/esp_new_jpeg/include/esp_jpeg_dec.h`
- [Preview Decoder](https://github.com/kun-acm/esp32s3-av-monitor/blob/main/main/APP/camera_preview.c): `main/APP/camera_preview.c`

### Network Services

- [Wi-Fi STA](https://github.com/kun-acm/esp32s3-av-monitor/blob/main/main/APP/wifi_sta.c): `main/APP/wifi_sta.c`
- [HTTP and MJPEG](https://github.com/kun-acm/esp32s3-av-monitor/blob/main/main/APP/av_stream.c): `main/APP/av_stream.c`
- [Audio WebSocket](https://github.com/kun-acm/esp32s3-av-monitor/blob/main/main/APP/audio_tcp.c): `main/APP/audio_tcp.c`

### Detection and Display

- [Face Detection Task](https://github.com/kun-acm/esp32s3-av-monitor/blob/main/main/APP/face_detect_task.c): `main/APP/face_detect_task.c`
- [MSR01 Detector](https://github.com/kun-acm/esp32s3-av-monitor/blob/main/main/APP/face_detect.cpp): `main/APP/face_detect.cpp`
- [LCD Video Renderer](https://github.com/kun-acm/esp32s3-av-monitor/blob/main/main/APP/lcd_video.c): `main/APP/lcd_video.c`
- [ILI9341 Driver](https://github.com/kun-acm/esp32s3-av-monitor/blob/main/main/APP/lcd_drv.c): `main/APP/lcd_drv.c`
- [LCD Status Overlay](https://github.com/kun-acm/esp32s3-av-monitor/blob/main/main/APP/lcd_display.c): `main/APP/lcd_display.c`
- [Motion Detection](https://github.com/kun-acm/esp32s3-av-monitor/blob/main/main/APP/motion_detect.c): `main/APP/motion_detect.c`

### Audio and Controls

- [I2S Audio Capture](https://github.com/kun-acm/esp32s3-av-monitor/blob/main/main/APP/audio_capture.c): `main/APP/audio_capture.c`
- [I2S Audio Playback](https://github.com/kun-acm/esp32s3-av-monitor/blob/main/main/APP/audio_playback.c): `main/APP/audio_playback.c`
- [μ-law Codec](https://github.com/kun-acm/esp32s3-av-monitor/blob/main/main/APP/mulaw.c): `main/APP/mulaw.c`
- [Light Controller](https://github.com/kun-acm/esp32s3-av-monitor/blob/main/main/APP/flash_light.c): `main/APP/flash_light.c`
- [Volume and Keys](https://github.com/kun-acm/esp32s3-av-monitor/blob/main/main/APP/volume_ctrl.c): `main/APP/volume_ctrl.c`

### Other components

- **LAN Browser**
- **OV5640 Camera**

## Connections

- LAN Browser → Wi-Fi STA: connects over LAN
- Camera Setup → OV5640 Camera: configures
- OV5640 Camera → JPEG Frame Cache: captures JPEG
- JPEG Frame Cache → Preview Decoder: copies frames
- Preview Decoder → JPEG Decoder API: decodes with (evidence: `main/APP/camera_preview.c`)
- JPEG Frame Cache → HTTP and MJPEG: serves frame copies (evidence: `main/APP/av_stream.c`)
- HTTP and MJPEG → LAN Browser: streams MJPEG and HTTP
- Preview Decoder → Face Detection Task: provides RGB preview
- Face Detection Task → MSR01 Detector: runs inference
- Face Detection Task → HTTP and MJPEG: publishes face results (evidence: `main/APP/av_stream.c`)
- Preview Decoder → LCD Video Renderer: provides RGB preview
- LCD Video Renderer → ILI9341 Driver: renders over SPI (evidence: `main/APP/lcd_video.c`)
- LCD Status Overlay → ILI9341 Driver: draws status (evidence: `main/APP/lcd_display.c`)
- I2S Audio Capture → Audio WebSocket: supplies captured audio (evidence: `main/APP/audio_tcp.c`)
- Audio WebSocket → μ-law Codec: encodes and decodes (evidence: `main/APP/audio_tcp.c`)
- Audio WebSocket → I2S Audio Playback: queues received audio (evidence: `main/APP/audio_tcp.c`)
- Audio WebSocket → LAN Browser: exchanges audio
- HTTP and MJPEG → Light Controller: handles light controls (evidence: `main/APP/av_stream.c`)
- Volume and Keys → I2S Audio Capture: controls audio level
- LAN Browser → HTTP and MJPEG: requests controls and results
- Motion Detection → HTTP and MJPEG