# thu-dsp-lab/ventus-gpgpu architecture

> Architecture diagram of the GitHub repository thu-dsp-lab/ventus-gpgpu, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-05-08). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/thu-dsp-lab/ventus-gpgpu
- Repository: https://github.com/thu-dsp-lab/ventus-gpgpu

## Overview

##### Project type and purpose
- Ventus is an open-source GPGPU processor built in Chisel (HDL), targeting RISC-V + vector extension and supporting OpenCL.
- It comprises: hardware modules (pipeline, caches, CTA scheduler), simulation frameworks (chiseltest, Verilator), an FPGA proof-of-concept flow, and host-side toolchain integration (LLVM/OpenCL compiler, ISA simulator, driver, test harness).

##### File structure
- Top-level build: `build.sc`/`common.sc`, `Makefile`, `shell.nix` for environment and build orchestration.
- `dependencies/…` holds submodule HDL libraries (Rocket-Chip, hardfloat, fpuv2).
- `ventus/src/…` contains Chisel code organized by subsystem:
- `pipeline` (fetch/decode/execute/issue/writeback, ALU, FPU, LSU, scoreboard, SIMT stack)
- `cta` (work-group scheduler, resource table, CU interface, warp‐buffer)
- `L1Cache` & `L2cache` (caches, arbiters, MSHRs, directory, banked SRAM template)
- `axi` (AXI4 adapters, AXI4Lite interfaces)
- `top` (wrappers: GPGPU_top, sim wrappers, external memory model, FakeCache, MemBox)
- `sim-verilator` implements a custom Verilator-based simulation harness.
- `ventus/fpga_test` has FPGA project scripts & drivers.
- `ventus/tests` are Scala unit tests (chiseltest) for submodules.
- `ventus/txt` contains kernel test cases (`.data` + `.metadata`) for host stimuli.

##### README
- Confirms a micro-architecture with a host toolchain (LLVM/OpenCL), ISA simulator, test driver, and the hardware design.

##### Main components
a. Host & Toolchain
- OpenCL front-end (LLVM) → binary + `.metadata`/.data → test driver / ISA simulator
b. Simulation & FPGA flows
- Chisel build → Verilator harness (`sim-verilator`)
- FPGA project scripts & constraints → Vivado (project_gpgpu.tcl) → bitstream
c. Hardware modules inside GPGPU_top
- Instruction fetch/decode & pipeline stages
- CTA scheduler (work-group dispatch)
- SIMD execution units (ALU, FPU, Vector units)
- Local memory (L1 DCache + ICache)
- Shared memory arbiter (atomic & bank conflict resolution)
- L2 cache (directory, MSHR, SRAM banks)
- AXI4Adapter connecting to external / global memory
d. External memory models
- Verilator wrapper memory (`ExternalMemModel`)
- FakeCache (for functional simulation)
- On FPGA: actual DDR via AXI4 wrapper (`GPGPU_axi_adapter`)

##### Relationships & data flow
- Host toolchain emits kernel binary → Driver/Test harness loads `.data` + `.metadata` into memory model.
- GPGPU_top ↔ AXI4Adapter ↔ L2 cache ↔ off-chip memory (or sim model).
- L2 ↔ L1 caches per SM → pipeline’s LSU/ICache → execution units.
- CTA scheduler issues workgroups into pipeline and allocates local/shared memory.
- Pipeline stages communicate via internal buses (issue → dispatch → ALU/FPU → writeback).

##### Architectural patterns & design principles
- Hierarchical modularity: separate caches, scheduler, pipeline.
- Standard interface patterns: AXI4/AXI4-Lite adapters for memory.
- Test harness abstraction: simulation wrapper vs FPGA wrapper.
- Parameterized Chisel modules (config files) for easy tuning.

##### Elements
- Module parameters (e.g., “L1 DCache: 32 kB, 2 ways”) if space permits.

## Diagram (Mermaid)

```mermaid
flowchart TD
    %% Host & Toolchain Layer
    subgraph "Host & Toolchain" 
        direction TB
        OCLOP["OpenCL FE (LLVM)"]:::toolchain
        TestHarness["Test Harness / ISA Simulator"]:::toolchain
        HostData["Kernel .data/.metadata"]:::toolchain
    end
    click HostData "https://github.com/thu-dsp-lab/ventus-gpgpu/tree/master/ventus/txt/"
    click TestHarness "https://github.com/thu-dsp-lab/ventus-gpgpu/tree/master/ventus/tests/"

    OCLOP -->|"compile kernel"| HostData
    HostData -->|"load stimuli"| TestHarness

    %% Simulation & FPGA Flow
    subgraph "Simulation & FPGA Flows"
        direction TB
        SimVer["Verilator Simulation"]:::sim
        FPGAFlow["FPGA Proof-of-Concept"]:::sim
    end
    click SimVer "https://github.com/thu-dsp-lab/ventus-gpgpu/blob/master/sim-verilator/sim_main.cpp"
    click SimVer "https://github.com/thu-dsp-lab/ventus-gpgpu/blob/master/sim-verilator/kernel.cpp"
    click SimVer "https://github.com/thu-dsp-lab/ventus-gpgpu/blob/master/sim-verilator/kernel.hpp"
    click SimVer "https://github.com/thu-dsp-lab/ventus-gpgpu/blob/master/sim-verilator/physical_mem.cpp"
    click SimVer "https://github.com/thu-dsp-lab/ventus-gpgpu/blob/master/sim-verilator/physical_mem.hpp"
    click SimVer "https://github.com/thu-dsp-lab/ventus-gpgpu/tree/master/sim-verilator/Makefile"
    click SimVer "https://github.com/thu-dsp-lab/ventus-gpgpu/blob/master/sim-verilator/verilate.mk"
    click FPGAFlow "https://github.com/thu-dsp-lab/ventus-gpgpu/blob/master/ventus/fpga_test/project_gpgpu.tcl"
    click FPGAFlow "https://github.com/thu-dsp-lab/ventus-gpgpu/blob/master/ventus/fpga_test/scrs/driver/naive_driver.c"
    click FPGAFlow "https://github.com/thu-dsp-lab/ventus-gpgpu/blob/master/ventus/fpga_test/scrs/driver/naive_driver.h"
    click FPGAFlow "https://github.com/thu-dsp-lab/ventus-gpgpu/blob/master/ventus/fpga_test/scrs/gpgpu_fpga_test/GPGPU_axi_adapter.v"
    click FPGAFlow "https://github.com/thu-dsp-lab/ventus-gpgpu/blob/master/ventus/fpga_test/scrs/constrs/toplevel.xdc"

    TestHarness --> SimVer
    TestHarness --> FPGAFlow

    %% Build & Env
    subgraph "Build & Environment" 
        direction TB
        BuildSC["build.sc"]:::build
        CommonSC["common.sc"]:::build
        MakefileF["Makefile"]:::build
        ShellNix["shell.nix"]:::build
    end
    click BuildSC "https://github.com/thu-dsp-lab/ventus-gpgpu/blob/master/build.sc"
    click CommonSC "https://github.com/thu-dsp-lab/ventus-gpgpu/blob/master/common.sc"
    click MakefileF "https://github.com/thu-dsp-lab/ventus-gpgpu/tree/master/Makefile"
    click ShellNix "https://github.com/thu-dsp-lab/ventus-gpgpu/blob/master/shell.nix"

    SimVer --> BuildSC
    FPGAFlow --> BuildSC

    %% Core Hardware
    subgraph "GPGPU_top (Chisel Module)" 
        direction TB
        GPGPU["GPGPU_top"]:::core
        subgraph "Top Wrappers & Models"
            direction TB
            SimWrap["GPGPU_SimWrapper"]:::interface
            ExtMemWrap["GPGPU_ExtMemWrapper"]:::interface
            MemBox["MemBox"]:::interface
            ExtModel["ExternalMemModel"]:::interface
            FakeCache["FakeCache"]:::interface
        end
        subgraph "CTA Scheduler (Control)" 
            direction TB
            CTASched["cta_scheduler"]:::control
            Allocator["allocator"]:::control
            ResTable["resource_table"]:::control
            CUIf["cu_interface"]:::control
            WgBuf["wg_buffer"]:::control
        end
        subgraph "Pipeline & Execution (Data Path)" 
            direction TB
            Decode["DecodeUnit"]:::datapath
            Issue["issue"]:::datapath
            ExecUnit["execution"]:::datapath
            WriteBack["writeback"]:::datapath
            ALU["ALU"]:::datapath
            FPU["FPU / FloatDivSqrt"]:::datapath
            LSU["LSU"]:::datapath
            SIMT["SIMT_STACK"]:::datapath
            Score["scoreboard"]:::datapath
        end
        subgraph "L1 Cache & Shared Memory (Memory)" 
            direction TB
            DCache["DCache"]:::memory
            ICache["ICache"]:::memory
            Atomic["AtomicUnit"]:::memory
            ShareMem["ShareMem"]:::memory
            ArbiterL1["L1Cache2L2Arbiter"]:::memory
            MSHR1["L1MSHR"]:::memory
        end
        subgraph "L2 Cache & Directory (Memory)" 
            direction TB
            Dir["Directory"]:::memory
            MSHR2["MSHR"]:::memory
            SRAM["SRAMTemplate"]:::memory
            Banked["BankedStore"]:::memory
        end
        AXI["AXI4Adapter"]:::interface
        GPGPU --> SimWrap
        GPGPU --> ExtMemWrap
        GPGPU --> MemBox
        GPGPU --> CTASched
        GPGPU --> Decode
        CTASched --> Issue
        Issue --> ALU
        Issue --> FPU
        ALU --> WriteBack
        FPU --> WriteBack
        WriteBack --> Score
        LSU --> SRAM
        DCache --> ArbiterL1
        ICache --> ArbiterL1
        ArbiterL1 --> SRAM
        SRAM --> AXI
        AXI -->|"AXI4 r/w"| Dir
        Dir --> MSHR2
        MSHR2 --> Banked
        Banked -->|"off-chip mem"| ExtModel
        FakeCache --> GPGPU
    end
    click GPGPU "https://github.com/thu-dsp-lab/ventus-gpgpu/blob/master/ventus/src/top/GPGPU_top.scala"
    click SimWrap "https://github.com/thu-dsp-lab/ventus-gpgpu/blob/master/ventus/src/top/GPGPU_SimWrapper.scala"
    click ExtMemWrap "https://github.com/thu-dsp-lab/ventus-gpgpu/blob/master/ventus/src/top/GPGPU_ExtMemWrapper.scala"
    click ExtModel "https://github.com/thu-dsp-lab/ventus-gpgpu/blob/master/ventus/src/top/ExternalMemModel.scala"
    click FakeCache "https://github.com/thu-dsp-lab/ventus-gpgpu/blob/master/ventus/src/top/FakeCache.scala"
    click MemBox "https://github.com/thu-dsp-lab/ventus-gpgpu/blob/master/ventus/src/top/MemBox.scala"
    click CTASched "https://github.com/thu-dsp-lab/ventus-gpgpu/blob/master/ventus/src/cta/cta_scheduler.scala"
    click Allocator "https://github.com/thu-dsp-lab/ventus-gpgpu/blob/master/ventus/src/cta/allocator.scala"
    click ResTable "https://github.com/thu-dsp-lab/ventus-gpgpu/blob/master/ventus/src/cta/resource_table.scala"
    click CUIf "https://github.com/thu-dsp-lab/ventus-gpgpu/blob/master/ventus/src/cta/cu_interface.scala"
    click WgBuf "https://github.com/thu-dsp-lab/ventus-gpgpu/blob/master/ventus/src/cta/wg_buffer.scala"
    click Decode "https://github.com/thu-dsp-lab/ventus-gpgpu/blob/master/ventus/src/pipeline/DecodeUnit.scala"
    click Issue "https://github.com/thu-dsp-lab/ventus-gpgpu/blob/master/ventus/src/pipeline/issue.scala"
    click ExecUnit "https://github.com/thu-dsp-lab/ventus-gpgpu/blob/master/ventus/src/pipeline/execution.scala"
    click WriteBack "https://github.com/thu-dsp-lab/ventus-gpgpu/blob/master/ventus/src/pipeline/writeback.scala"
    click ALU "https://github.com/thu-dsp-lab/ventus-gpgpu/blob/master/ventus/src/pipeline/ALU.scala"
    click FPU "https://github.com/thu-dsp-lab/ventus-gpgpu/blob/master/ventus/src/pipeline/FPU.scala"
    click LSU "https://github.com/thu-dsp-lab/ventus-gpgpu/blob/master/ventus/src/pipeline/LSU.scala"
    click SIMT "https://github.com/thu-dsp-lab/ventus-gpgpu/blob/master/ventus/src/pipeline/SIMT_STACK.scala"
    click Score "https://github.com/thu-dsp-lab/ventus-gpgpu/blob/master/ventus/src/pipeline/scoreboard.scala"
    click DCache "https://github.com/thu-dsp-lab/ventus-gpgpu/blob/master/ventus/src/L1Cache/DCache/DCache.scala"
    click ICache "https://github.com/thu-dsp-lab/ventus-gpgpu/blob/master/ventus/src/L1Cache/ICache/ICache.scala"
    click Atomic "https://github.com/thu-dsp-lab/ventus-gpgpu/blob/master/ventus/src/L1Cache/AtomicUnit/AtomicUnit.scala"
    click ShareMem "https://github.com/thu-dsp-lab/ventus-gpgpu/blob/master/ventus/src/L1Cache/ShareMem/ShareMem.scala"
    click ArbiterL1 "https://github.com/thu-dsp-lab/ventus-gpgpu/blob/master/ventus/src/L1Cache/L1Cache2L2Arbiter.scala"
    click MSHR1 "https://github.com/thu-dsp-lab/ventus-gpgpu/blob/master/ventus/src/L1Cache/L1MSHR.scala"
    click Dir "https://github.com/thu-dsp-lab/ventus-gpgpu/blob/master/ventus/src/L2cache/Directory_test.scala"
    click MSHR2 "https://github.com/thu-dsp-lab/ventus-gpgpu/blob/master/ventus/src/L2cache/MSHR.scala"
    click SRAM "https://github.com/thu-dsp-lab/ventus-gpgpu/blob/master/ventus/src/L2cache/SRAMTemplate.scala"
    click Banked "https://github.com/thu-dsp-lab/ventus-gpgpu/blob/master/ventus/src/L2cache/BankedStore.scala"
    click AXI "https://github.com/thu-dsp-lab/ventus-gpgpu/blob/master/ventus/src/axi/AXI4Adapter.scala"

    %% Styles
    classDef toolchain fill:#D6E4FF,stroke:#1F4E79;
    classDef sim fill:#F0F4C3,stroke:#7C4DFF;
    classDef build fill:#E0E0E0,stroke:#424242;
    classDef core fill:#ECEFF1,stroke:#263238;
    classDef control fill:#BBDEFB,stroke:#0D47A1;
    classDef datapath fill:#C8E6C9,stroke:#1B5E20;
    classDef memory fill:#FFE0B2,stroke:#E65100;
    classDef interface fill:#CFD8DC,stroke:#455A64;
```
