How kun-acm/esp32s3-av-monitor works
This is ESP32-S3 firmware for LAN audio/video monitoring, local face detection, and an LCD preview. The principal workflow captures OV5640 JPEG frames into a latest-frame cache, serves MJPEG to a browser, and separately decodes frames for face detection and LCD display. Audio capture/playback uses a distinct WebSocket path; HTTP also exposes status, snapshots, face results, and light controls. Relationships follow the README’s documented data flow and the sampled startup and handler code; several subsystem implementations were not excerpted, so uncertain internal calls are omitted.
Main entry points: av_stream.c HTTP and MJPEG · camera_preview.c Preview Decoder · audio_tcp.c Audio WebSocket · camera_frames.c JPEG Frame Cache
Components 20
Camera Pipeline
- Camera Setup
main/APP/camera_init.c - JPEG Frame Cache
main/APP/camera_frames.c - JPEG Decoder API
components/esp_new_jpeg/include/esp_jpeg_dec.h - Preview Decoder
main/APP/camera_preview.c
Network Services
- Wi-Fi STA
main/APP/wifi_sta.c - HTTP and MJPEG
main/APP/av_stream.c - Audio WebSocket
main/APP/audio_tcp.c
Detection and Display
- Face Detection Task
main/APP/face_detect_task.c - MSR01 Detector
main/APP/face_detect.cpp - LCD Video Renderer
main/APP/lcd_video.c - ILI9341 Driver
main/APP/lcd_drv.c - LCD Status Overlay
main/APP/lcd_display.c - Motion Detection
main/APP/motion_detect.c
Audio and Controls
- I2S Audio Capture
main/APP/audio_capture.c - I2S Audio Playback
main/APP/audio_playback.c - μ-law Codec
main/APP/mulaw.c - Light Controller
main/APP/flash_light.c - Volume and Keys
main/APP/volume_ctrl.c
Other components
- LAN Browser
- OV5640 Camera
Connections 21
- LAN Browser → Wi-Fi STA: connects over LAN
- Camera Setup → OV5640 Camera: configures
- OV5640 Camera → JPEG Frame Cache: captures JPEG
- JPEG Frame Cache → Preview Decoder: copies frames
- Preview Decoder → JPEG Decoder API: decodes with
main/APP/camera_preview.c - JPEG Frame Cache → HTTP and MJPEG: serves frame copies
main/APP/av_stream.c - HTTP and MJPEG → LAN Browser: streams MJPEG and HTTP
- Preview Decoder → Face Detection Task: provides RGB preview
- Face Detection Task → MSR01 Detector: runs inference
- Face Detection Task → HTTP and MJPEG: publishes face results
main/APP/av_stream.c - Preview Decoder → LCD Video Renderer: provides RGB preview
- LCD Video Renderer → ILI9341 Driver: renders over SPI
main/APP/lcd_video.c - LCD Status Overlay → ILI9341 Driver: draws status
main/APP/lcd_display.c - I2S Audio Capture → Audio WebSocket: supplies captured audio
main/APP/audio_tcp.c - Audio WebSocket → μ-law Codec: encodes and decodes
main/APP/audio_tcp.c - Audio WebSocket → I2S Audio Playback: queues received audio
main/APP/audio_tcp.c - Audio WebSocket → LAN Browser: exchanges audio
- HTTP and MJPEG → Light Controller: handles light controls
main/APP/av_stream.c - Volume and Keys → I2S Audio Capture: controls audio level
- LAN Browser → HTTP and MJPEG: requests controls and results
- Motion Detection → HTTP and MJPEG