How markcxli/fpga_dpu works
Project Type and Purpose
• This is a hardware-software co-design project that accelerates a state-of-the-art multiple-object-detection model (YOLOv2) on both GPU and FPGA. The host side runs the Darknet framework (C/C++ + CUDA) for inference, and a custom DPU (Deep Learning Processor Unit) on a Xilinx FPGA (via DNNDK) to measure performance and power/efficiency trade-offs.
File-Structure Analysis
• Top-level directories:
• YOLO/src – Core Darknet code (C files, CUDA kernels, headers)
• YOLO/cfg – Model configuration files (network definitions, dataset specs)
• YOLO/data – Class names, label mappings, helper scripts for data prep
• YOLO/examples – Example inference programs in C and Python
• YOLO/python – Python wrappers (darknet.py) and utilities
• YOLO/scripts – Shell/Python scripts for dataset downloading and label generation
• YOLO/include – Public header (darknet.h)
• YOLO/libdarknet.a, YOLO/obj – Compiled library and object files
• Key build and deployment files: Makefile for host compilation; DNNDK APIs integrated in refactored code; Vivado-generated bitstream not in tree but implied by README flow.
README Insights
• Describes the Xilinx DPU IP and the DNNDK development flow: Vivado → bitstream → DPU driver → host application.
a. Main Components to Represent
• Host Application Layer
• Darknet Inference Engine (darknet.c, parser, network modules)
• Model Definitions (cfg/*.cfg, *.weights)
• DNNDK API Wrapper (refactored C using dnn_native_load, dpu_run)
• GPU Inference Path
• CUDA Kernels (src/*_kernels.cu)
• NVIDIA GPU + Device Memory
• FPGA Inference Path
• DPU IP on FPGA (PL logic: scheduler, compute array, global memory)
• Vivado-generated bitstream loaded into FPGA fabric
• DNNDK Driver in PS (processing system)
• PCIe / AXI interconnect + DDR memory
• Data Flow and Measurements
• Input Image Source (camera / file system)
• Performance Monitor (roofline, bandwidth calculator)
b. Relationships & Interactions
• Host App loads model + image → invokes either CUDA kernels or DNNDK calls
• GPU branch: host ↔ CUDA runtime ↔ GPU device memory ↔ CUDA kernels
• FPGA branch: host ↔ DPU driver ↔ PCIe/AXI ↔ DPU hardware ↔ DDR memory
• Results flow back to host, then to visualization or logging scripts
c. Architectural Patterns & Principles
• Accelerator Offload Pattern (host delegates heavy convolution workloads to GPU/FPGA)
• Modular Layered Design (parser, network, layers, utils separated in source)
• Hardware-Software Co-Design (C code + DNNDK API interfacing PL logic)
d. Key Technologies/Frameworks
• Darknet (C/C++ object-detection engine)
• CUDA (NVIDIA GPU kernels)
• Xilinx DPU IP (Vivado, bitstream)
• DNNDK (dnn_native_load, dpu_run, device driver)
• Python for scripting & prototyping (darknet.py)