How georgeyhere/fpga-video-processing works
Overall Project Identification
• The project is a hardware design for an FPGA (Artix-7) that implements a video processing pipeline. It captures video via an OV7670 camera, applies filtering (Gaussian lowpass and Sobel edge detection), and outputs data through an HDMI interface.
• The design mixes digital hardware (Verilog RTL) with FPGA IP cores (e.g., clock wizard under the ip directory) and test benches for simulation (under tb).
Analyzing the File Structure
• Top-level directories include:
• "ip": Houses the clock management IP core (clk_wiz_0) with various Vivado files, configurations, and board constraints.
• "rtl": Contains all of the hardware modules – camera interfaces, HDMI modules, processing modules (greyscale conversion, Gaussian and Sobel filtering kernels), asynchronous/synchronous FIFOs, memory interfaces, and control logic.
• "tb": Contains simulation test benches to validate each module.
• This grouping shows that the design is modular with each processing stage implemented as separate Verilog modules and IP cores.
Extracting Key Components from the README
• The README explains the system is logically divided into:
• Camera Interface: Configuring the OV7670 camera using I²C and capturing pixel data into asynchronous FIFOs.
• Preprocessing: Converting the incoming video data from RGB444 to 8-bit greyscale, using a bit-shift based approximate algorithm (y = 0.299R + 0.587G + 0.114B).
• Kernel Processing: Two main processing pipelines – one for a Gaussian filter and one for a Sobel edge detection. A common kernel control module (ps_kernel_control) organizes line buffers and feeds pixel clusters into the kernel pipelines.
• Memory/Output and Display: Processed data is stored in block RAM (framebuffer) and then accessed by the HDMI transmitter, with conversion from greyscale to a required output format (extension to RGB888 for TMDS encoding). Also includes logic for RGB passthrough mode.
Key Components
• External Interfaces:
• OV7670 Camera (including I²C configuration) feeding into the system.
• HDMI Output interface.
• User Input (buttons and switches on the dev board).
• Core Processing Blocks (modules):
• Camera Interface Block: Captures and formats RGB444 video from the OV7670, loading data into an asynchronous FIFO.
• Preprocessing Block: Converts the RGB444 video input to eight-bit greyscale using the provided algorithm.
• Kernel Control & Filtering Blocks:
• Kernel Control Module (ps_kernel_control): Manages line buffers and pixel data extraction.
• Gaussian Filter Module: Implements a Gaussian lowpass filter.
• Sobel Filter Module: Implements edge detection (including thresholding, avoiding square root computations).
• Memory and Display Interface:
• Framebuffer Block: Synthesized block memory used for temporary pixel storage.
• HDMI Transmitter: Converts processed pixel data for display (including RGB444 to TMDS conversion if in passthrough mode, or RGB888 conversion for processed video).
• Support/Utility Components:
• Clock Wizard (under ip): Essential FPGA clock generation and management for timing across modules.
• FIFOs and Memory Interfaces: For asynchronous and synchronous data buffering between stages.
Relationships and Data Flow
• From the Camera Interface to Preprocessing (data flow of raw RGB444 video).
• From Preprocessing to Kernel Control (passing greyscale pixel data).
• From Kernel Control to both the Gaussian and Sobel Filtering pipelines (feeding sliding window pixel data).
• From Filtering pipelines to the Framebuffer (synchronized write/read).
• Finally from the Framebuffer to the HDMI Transmitter.
• User interaction feedback (e.g., button inputs selecting video modes and filter thresholds) interfacing with control blocks.
Architectural Patterns and Design Principles
• Modular design: Each pipeline stage is treated as an independent module with clearly defined inputs and outputs.
• Pipelining: Streaming data through sequential blocks (camera capture → preprocessing → filtering → output) to maintain throughput and meet real-time video constraints.
• Data buffering: Use of asynchronous/synchronous FIFOs and block RAM to handle timing differences across modules.
• Hardware-specific IP integration: Incorporation of vendor IP (like the clk_wiz_0) to abstract clock management, emphasizing separation of concerns.