How jpwright/fpganes works
Project Type and Purpose
• This is a hardware‐only, FPGA‐based image pipeline and AI “agent” for playing Super Mario Bros. on the Altera DE2 board (no soft CPU or operating system). It streams video in from a camera (ITU‐656), processes it through custom Verilog “AI” logic, stores frames in SDRAM, and outputs video to VGA plus audio via an I²C‐configured DAC. MATLAB and Excel scripts are used offline to generate parameterized buffers and sprite‐LUTs baked into the bitstream.
File‐Structure Analysis
• Top‐level:
• cam_to_vga/ – Quartus project + all Verilog modules
• matlab_generators/ – .m scripts to auto‐generate circular buffers and update Verilog parameters
• excel_generators/ – XLSX that outputs Verilog for sprites/kernels
• schematic/ – DE2 hardware schematic for reference
• Within cam_to_vga/:
• DE2_TV.* – Quartus design files, fit/map reports, pin assignments
• *.v – Verilog modules (AUDIO_DAC, I2C_AV_Config, ITU_656_Decoder, YUV422_to_444, YCbCr2RGB, line buffers, SDRAM controller, VGA controllers, detection modules, goomba.v, recolor.v, intensityCalc.v, etc.)
• Sdram_Control_4Port/ – multi‐port SDRAM controller IP and FIFO interfaces
• *.mif, *.txt – pre‐computed lookup tables (RGB, segment LUTs, etc.)
• script.m – top‐level MATLAB driver to regenerate buffers
README Insights
• The entry point is DE2_TV.v, which instantiates and wires together all submodules.
• MATLAB/Excel are purely offline code‐gen steps that produce MIF files and Verilog includes; they are not part of the FPGA bitstream’s run‐time.
a. Main Components
• Camera Sensor + I²C AV Configuration block
• ITU_656_Decoder → YUV422_to_444 → YCbCr2RGB converter
• Line_Buffer (stream pipeline)
• Sdram_Control_4Port (frame read/write memory)
• AI/Image‐processing modules: recolor.v, intensityCalc.v, detector.v, edgedetectH/V.v, goomba.v, pipe_corner.v, etc.
• VGA_Ctrl / VGA_Controller (fetch from SDRAM & generate timing)
• Audio path: VGA_Audio_PLL → AUDIO_DAC (I²S or parallel audio)
• Offline Code‐Generation: MATLAB scripts & Excel sprite generator feeding MIFs into compile-time
b. Relationships & Data Flows
[Camera] → [I²C_AV_Config] → [ITU_656_Decoder] → [YUV→RGB chain] →
→ Stream into [Line_Buffer] → either direct pixel ops or write to [SDRAM Controller]
→ Processing modules read/write SDRAM → final frames read by [VGA_Controller] → [VGA].
• Audio branch taps off PLL into AUDIO_DAC.
• Offline scripts produce lookup‐table files (MIFs) which are bundled into the bitstream but sit outside the run‐time path.
c. Architectural Patterns & Principles
• Streaming pipeline + multi-port shared memory (SDRAM) for frame buffering
• Modular design—each image function is a separate Verilog module with standard streaming FIFO interfaces
• Offline code generation to keep on-FPGA logic generic and parameterized
d. Key Technologies & IP
• Verilog HDL targeting Altera/Intel Cyclone II (DE2 board)
• Altera SDRAM IP (Sdram_Control_4Port)
• I²C AV config core
• PLL blocks for clocking video/audio
• MATLAB & Excel for LUT/MIF generation
• Quartus compilation flow (db/, incremental_db/ folders)