How romanlut/hx-esp32-cam-fpv works
Project Type and Purpose
• This is an open-source digital FPV (First-Person View) system. It combines embedded firmware running on ESP32/ESP32-S3 based air units (cameras + codecs + telemetry) with cross-platform ground station (GS) software (Wi-Fi packet capture, FEC decode, JPEG decode, OSD, recording).
• Purpose: capture MJPEG frames on an OV2640/OV5640 sensor, FEC-encode and inject them over Wi-Fi, decode and render on a low-cost ground station (Radxa Zero or Raspberry Pi variants) with bi-directional telemetry and RC control.
File-Structure Analysis
• Top-Level Workflows
• .github/workflows: CI builds for multiple firmware targets and releases.
• scripts/: host-side helpers (boot selection, fan control, SD imaging, installation).
• Air-Unit Firmware
• air_firmware_esp32cam / air_firmware_esp32s3sense / air_firmware_esp32s3sense_ov5640
– Each variant has its own CMakeLists, PlatformIO project, sdkconfig, partitions.csv, upload script.
– main/ contains dummy.cpp placeholder – actual logic lives in components/*
• Shared Components
• components/common: JPEG parser, FEC encoder/decoder, MAVLink parser, circular buffers, packet filters, utility libraries.
• components/air: HTTP file server, OSD generator, NVS settings, Wi-Fi setup, temperature sensor, queue abstraction, MSP/MAVLink, Python test server, upload script HTML.
• components/esp32-camera: Forked IDF camera driver, conversion routines, sensor-specific code, DMA streaming modifications.
• Ground Station (gs/)
• C++ application with src/: networking (packet capture via libpcap), FEC decode, TurboJPEG decode, OpenGL ES rendering of textures + ImGui based OSD menu, SDL/GLFW window, MAVLink/RC uplink.
• assets/osd_fonts: font PNGs for on-screen overlay.
• launch.sh: script to detect wlan interfaces, set monitor mode, start GS binary.
• Documentation (doc/)
• Flashing guides, hardware pinouts, datasheets, deployment/release instructions.
Key Architectural Insights from README
• Pipeline on Air: Sensor → DMA JPEG stream → (optional) SD buffer → FEC encode → Wi-Fi injection
• Pipeline on GS: Monitor-mode Wi-Fi capture → FEC decode → JPEG data → TurboJPEG decode → Texture upload + OpenGL ES render + OSD overlays → (optional) file write.
• Bidirectional serial link over Wi-Fi for MAVLink/RC via FEC.
• Adaptive JPEG compression controlling frame size based on queue occupancy and link capacity.
• Multi-adapter diversity on GS for reception robustness.
Diagramming the System
a. Main Components
• Air Unit Hardware
– OV2640 or OV5640 camera sensor
– ESP32 or ESP32-S3 Sense MCU
– SD card interface (DVR)
– GPIO Joystick, LEDs, REC button
– External services: Wi-Fi link (packet injection in monitor mode)
• Air Unit Firmware
– esp32-camera component (DMA JPEG streaming)
– Air logic component (adaptive compression, FEC encoder, SD recorder, queue manager, OSD generator, HTTP file server)
– Common libraries (MAVLink, CRC, JPEG parser, utilities)
• Wireless Layer
– 2.4 GHz Wi-Fi in monitor/packet-injection mode, FEC-encoded blocks
• Ground Station Hardware
– Radxa Zero 3W or Raspberry Pi + dual RTL8812AU cards
– HDMI/Composite output / FPV glasses
– Joystick + buttons
– SD card for recording
• Ground Station Software
– Launch script (wlan1/wlan2 detection, monitor mode setup)
– GS binary (C++): packet capture, FEC decode, TurboJPEG, OpenGL ES renderer, ImGui OSD menu, file writer, MAVLink uplink.
b. Relationships & Data Flows
1. Camera sensor → JPEG frames via I2S/DMA → Firmware
2. Firmware → Sequence of pipeline queues → FEC encode → Wi-Fi packet injection
3. Ground station Wi-Fi cards in monitor mode → raw 802.11 frames → packet capture
4. FEC decode → Reassemble JPEG frames → TurboJPEG decode → GPU texture → display + on-screen overlay
5. Ground station UI input (joystick/keyboard) → MAVLink/OSD UART → FEC encode → Wi-Fi injection → Air unit UART
6. Both sides optionally write .avi (MJPEG) to SD card.
c. Architectural Patterns & Principles
• Component-Based Design: IDF components for modularity (camera, common, air).
• Pipeline / Stream Processing: Queues between capture, encode, transmit stages.
• Cross-Platform Host App: CMake + C++17 for Linux, uses SDL/OpenGL ES, FFMPEG & pcap libs.
• Adaptive Feedback Control: dynamic JPEG quality based on queue occupancy & link metrics.
• FEC Without Retransmission: high-redundancy forward error correction for constant-bitrate streaming.
d. Key Technologies / Frameworks
• Embedded: ESP-IDF (CMake, component.mk), PlatformIO, FreeRTOS, I2S DMA, NVS
• Image: JPEG row-by-row, TurboJPEG
• Networking: Wi-Fi monitor mode, packet injection, pcap
• Graphics: OpenGL ES for texture upload + OSD, ImGui for menus
• Misc: MAVLink, CRC, circular buffer, FEC codec (K/N blocks), FATFS SD card, CMake for ground station.