How thu-dsp-lab/ventus-gpgpu works
Project type and purpose
• Ventus is an open-source GPGPU processor built in Chisel (HDL), targeting RISC-V + vector extension and supporting OpenCL.
• It comprises: hardware modules (pipeline, caches, CTA scheduler), simulation frameworks (chiseltest, Verilator), an FPGA proof-of-concept flow, and host-side toolchain integration (LLVM/OpenCL compiler, ISA simulator, driver, test harness).
File structure
• Top-level build: build.sc/common.sc, Makefile, shell.nix for environment and build orchestration.
• dependencies/… holds submodule HDL libraries (Rocket-Chip, hardfloat, fpuv2).
• ventus/src/… contains Chisel code organized by subsystem:
• pipeline (fetch/decode/execute/issue/writeback, ALU, FPU, LSU, scoreboard, SIMT stack)
• cta (work-group scheduler, resource table, CU interface, warp‐buffer)
• L1Cache & L2cache (caches, arbiters, MSHRs, directory, banked SRAM template)
• axi (AXI4 adapters, AXI4Lite interfaces)
• top (wrappers: GPGPU_top, sim wrappers, external memory model, FakeCache, MemBox)
• sim-verilator implements a custom Verilator-based simulation harness.
• ventus/fpga_test has FPGA project scripts & drivers.
• ventus/tests are Scala unit tests (chiseltest) for submodules.
• ventus/txt contains kernel test cases (.data + .metadata) for host stimuli.
README
• Confirms a micro-architecture with a host toolchain (LLVM/OpenCL), ISA simulator, test driver, and the hardware design.
Main components
a. Host & Toolchain
• OpenCL front-end (LLVM) → binary + .metadata/.data → test driver / ISA simulator
b. Simulation & FPGA flows
• Chisel build → Verilator harness (sim-verilator)
• FPGA project scripts & constraints → Vivado (project_gpgpu.tcl) → bitstream
c. Hardware modules inside GPGPU_top
• Instruction fetch/decode & pipeline stages
• CTA scheduler (work-group dispatch)
• SIMD execution units (ALU, FPU, Vector units)
• Local memory (L1 DCache + ICache)
• Shared memory arbiter (atomic & bank conflict resolution)
• L2 cache (directory, MSHR, SRAM banks)
• AXI4Adapter connecting to external / global memory
d. External memory models
• Verilator wrapper memory (ExternalMemModel)
• FakeCache (for functional simulation)
• On FPGA: actual DDR via AXI4 wrapper (GPGPU_axi_adapter)
Relationships & data flow
• Host toolchain emits kernel binary → Driver/Test harness loads .data + .metadata into memory model.
• GPGPU_top ↔ AXI4Adapter ↔ L2 cache ↔ off-chip memory (or sim model).
• L2 ↔ L1 caches per SM → pipeline’s LSU/ICache → execution units.
• CTA scheduler issues workgroups into pipeline and allocates local/shared memory.
• Pipeline stages communicate via internal buses (issue → dispatch → ALU/FPU → writeback).
Architectural patterns & design principles
• Hierarchical modularity: separate caches, scheduler, pipeline.
• Standard interface patterns: AXI4/AXI4-Lite adapters for memory.
• Test harness abstraction: simulation wrapper vs FPGA wrapper.
• Parameterized Chisel modules (config files) for easy tuning.
Elements
• Module parameters (e.g., “L1 DCache: 32 kB, 2 ways”) if space permits.