# jvhs0706/zkllm-ccs2024 architecture

> Architecture diagram of the GitHub repository jvhs0706/zkllm-ccs2024, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-04-03). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/jvhs0706/zkllm-ccs2024
- Repository: https://github.com/jvhs0706/zkllm-ccs2024

## Overview

##### Project Type & Purpose
- This is a specialized system showing both the cryptographic and deep learning components unified in a single CUDA-based codebase.
- The overall goal is to compute fixed-point representations of model parameters, generate corresponding commitments, and produce layered proofs for neural network/LLM computations.
- The system orchestrates GPU-accelerated computation (via CUDA routines) alongside Python scripts that handle model downloading, public parameter generation, and proof assembly.

##### Analyzing the File Structure
- There is a mix of CUDA source files (.cu, .cuh) that implement arithmetic over the BLS12-381 curve, tensor operations, and core LLM components (like self-attention, RMSNorm, skip connections, FFN, and polynomial calculations).
- Standalone Python scripts (e.g., download-models.py, llama-ppgen.py, llama-commit.py, llama-ffn.py, etc.) serve as higher-level orchestrators that invoke these CUDA kernels. They deal with tasks such as model downloading, parameter commitment generation, and managing layer proofs.
- Supporting files like Makefile, timer.cpp/ timer.hpp indicate build and timing management for performance benchmarking.
- The naming conventions indicate repeated patterns (e.g., similar CUDA file names for each component such as self-attn, skip-connection, and rmsnorm) which map directly to operational blocks in an LLM layer’s proof computation.

##### Key Components
a. Front-end / Orchestration Scripts:
- Python scripts: download-models.py (manages model acquisition), llama-ppgen.py (generates public parameters), llama-commit.py (produces fixed-point commitments), and various llama-*.py components that run specific steps for each layer.
b. CUDA Kernels & Components:
- Core computation modules (.cu/.cuh) such as:
- Elliptic curve arithmetic (bls12-381.cu, bls12-381.cuh)
- Tensor operations (e.g., g1-tensor, fr-tensor implementations)
- Proof computations (proof.cu, proof.cuh)
- Specialized functions for LLM layer operations (rmsnorm.cu, self-attn.cu, ffn.cu, skip-connection.cu, ppgen.cu)
- Dedicated modules for other cryptographic primitives (zkfc.cu, zkrelu.cu, zksoftmax.cu, tlookup.cu)
c. Data Management & I/O:
- File input/output utilities (fileio_utils.py, ioutils.cu/cuh) handling the reading/writing of fixed-point tensor data (.bin files).
d. Auxiliary Components:
- Timer utilities built in C++ (timer.cpp, timer.hpp) for performance measurements.
- Build and configuration via Makefile (specifying CUDA architectures like sm_86 for NVIDIA cards).

##### Relationships and Interactions
- How the public parameter generation (via llama-ppgen.py) is used as an input for the commitment process (llama-commit.py) to convert floating-point to fixed-point values.
- Sequential processing pipeline for each LLM layer:
i. Input preparation and RMSNorm – using llama-rmsnorm.py.
ii. Self-attention computations – using llama-self-attn.py.
iii. Skip connection – using llama-skip-connection.py.
iv. Feed-forward layer (FFN) – using llama-ffn.py.
- External dependencies where required (for instance, torch, transformers, and datasets, even if they’re not heavily integrated with the CUDA code, they provide context regarding the model’s ecosystem).

##### Architectural Patterns & Design Principles
- The design leverages the separation of concerns by isolating:
- Cryptographic routines (elliptic curve operations, commitment generation).
- Transformer component computations (via modular functions per network block).
- A coupling between numerical precision management (fixed-point conversions) and the corresponding proof generation.
- Python wrapper/orchestration components
- CUDA computational kernels (lower-level arithmetic and tensor operations)
- Data I/O & utility components

##### Elements to be Included
- All major components should be clearly labeled:
- "Model Downloader" (download-models.py)
- "Public Parameter Generator" (llama-ppgen.py)
- "Commitment Generator" (llama-commit.py)
- "Layer Processing Modules" (each of llama-rmsnorm.py, llama-self-attn.py, llama-skip-connection.py, llama-ffn.py)
- "CUDA Kernel Modules" (group similar CUDA files into blocks like "Elliptic Curve Arithmetic", "Tensor Operations", "ZK Proof Modules")
- "Utility and I/O Components" (fileio_utils.py, ioutils.cu/cuh)
- "Performance Monitoring" (timer.cpp/timer.hpp)
- Data flow: e.g., model parameters flow into “Commitment Generator” then distributed into each layer’s processing.
- Control flow: Python scripts invoking successive CUDA operations.

Happy diagramming!

## Diagram (Mermaid)

```mermaid
flowchart TD
    %% Orchestration Layer
    subgraph "Orchestration Layer"
        MD["Model Downloader"]:::orchestration
        PPG["Public Parameter Generator"]:::orchestration
        CG["Commitment Generator"]:::orchestration
        RNS["RMSNorm Processing"]:::orchestration
        SA["Self-Attention Processing"]:::orchestration
        SC["Skip Connection Processing"]:::orchestration
        FFN["Feed-Forward Network Processing"]:::orchestration
    end

    %% Orchestration CUDA Kernels
    subgraph "Orchestration CUDA Kernels"
        PP_CUDA["Public Param Kernel (ppgen.cu)"]:::cuda
        CG_CUDA1["Commit Param Kernel (commit-param.cu)"]:::cuda
        CG_CUDA2["Commitment Kernel (commitment.cu/cuh)"]:::cuda
        RMS_CUDA["RMSNorm Kernel (rmsnorm.cu)"]:::cuda
        SA_CUDA["Self-Attn Kernel (self-attn.cu)"]:::cuda
        SC_CUDA["Skip Connection Kernel (skip-connection.cu)"]:::cuda
        FFN_CUDA["FFN Kernel (ffn.cu)"]:::cuda
    end

    %% CUDA Core Operations Node
    CUDA_Core["CUDA Core Ops"]:::cuda

    %% CUDA Kernel Modules: Arithmetic & Tensor Ops
    subgraph "Arithmetic & Tensor Ops"
        ECC["Elliptic Curve Arithmetic (bls12-381.cu/cuh)"]:::cuda
        FR_Tensor["FR Tensor Ops (fr-tensor.cu/cuh)"]:::cuda
        G1_Tensor["G1 Tensor Ops (g1-tensor.cu/cuh)"]:::cuda
    end

    %% CUDA Kernel Modules: ZK Proof & Specialized Functions
    subgraph "ZK Proof & Specialized Functions"
        Proof["Proof Kernels (proof.cu/cuh)"]:::cuda
        Polynomial["Polynomial Kernels (polynomial.cu/cuh)"]:::cuda
        Rescaling["Rescaling Kernels (rescaling.cu/cuh)"]:::cuda
        ZKFC["zkfc (zkfc.cu/cuh)"]:::cuda
        ZKReLU["zkrelu (zkrelu.cu/cuh)"]:::cuda
        ZKSoftmax["zksoftmax (zksoftmax.cu/cuh)"]:::cuda
        TLookup["tlookup (tlookup.cu/cuh)"]:::cuda
    end

    %% Utility & I/O Components
    subgraph "Utility & I/O"
        IO_Py["File I/O Utils (fileio_utils.py)"]:::io
        IO_CUDA["I/O Utils CUDA (ioutils.cu/cuh)"]:::io
    end

    %% Performance Monitoring & Build
    subgraph "Performance & Build"
        Timer["Timer Utilities (timer.cpp/timer.hpp)"]:::performance
        Make["Build Configuration (Makefile)"]:::performance
    end

    %% Relationships between Orchestration Python Scripts
    MD --> PPG
    PPG --> CG
    CG --> RNS
    CG --> SA
    CG --> SC
    CG --> FFN

    %% Relationships from Orchestration to their CUDA Kernels
    PPG --> PP_CUDA
    CG --> CG_CUDA1
    CG --> CG_CUDA2
    RNS --> RMS_CUDA
    SA --> SA_CUDA
    SC --> SC_CUDA
    FFN --> FFN_CUDA

    %% Arrows from Orchestration CUDA Kernels to CUDA Core Ops
    PP_CUDA --> CUDA_Core
    CG_CUDA1 --> CUDA_Core
    CG_CUDA2 --> CUDA_Core
    RMS_CUDA --> CUDA_Core
    SA_CUDA --> CUDA_Core
    SC_CUDA --> CUDA_Core
    FFN_CUDA --> CUDA_Core

    %% Distribution from CUDA Core Ops to Core CUDA Modules
    CUDA_Core --> ECC
    CUDA_Core --> FR_Tensor
    CUDA_Core --> G1_Tensor
    CUDA_Core --> Proof
    CUDA_Core --> Polynomial
    CUDA_Core --> Rescaling
    CUDA_Core --> ZKFC
    CUDA_Core --> ZKReLU
    CUDA_Core --> ZKSoftmax
    CUDA_Core --> TLookup

    %% Additional Links for Data I/O and Performance Monitoring
    MD --> IO_Py
    CUDA_Core --> Timer
    Timer --> Make

    %% Click Events for Orchestration Layer
    click MD "https://github.com/jvhs0706/zkllm-ccs2024/blob/main/download-models.py"
    click PPG "https://github.com/jvhs0706/zkllm-ccs2024/blob/main/llama-ppgen.py"
    click CG "https://github.com/jvhs0706/zkllm-ccs2024/blob/main/llama-commit.py"
    click RNS "https://github.com/jvhs0706/zkllm-ccs2024/blob/main/llama-rmsnorm.py"
    click SA "https://github.com/jvhs0706/zkllm-ccs2024/blob/main/llama-self-attn.py"
    click SC "https://github.com/jvhs0706/zkllm-ccs2024/blob/main/llama-skip-connection.py"
    click FFN "https://github.com/jvhs0706/zkllm-ccs2024/blob/main/llama-ffn.py"

    %% Click Events for Orchestration CUDA Kernels
    click PP_CUDA "https://github.com/jvhs0706/zkllm-ccs2024/blob/main/ppgen.cu"
    click CG_CUDA1 "https://github.com/jvhs0706/zkllm-ccs2024/blob/main/commit-param.cu"
    click CG_CUDA2 "https://github.com/jvhs0706/zkllm-ccs2024/blob/main/commitment.cu/commitment.cuh"
    click RMS_CUDA "https://github.com/jvhs0706/zkllm-ccs2024/blob/main/rmsnorm.cu"
    click SA_CUDA "https://github.com/jvhs0706/zkllm-ccs2024/blob/main/self-attn.cu"
    click SC_CUDA "https://github.com/jvhs0706/zkllm-ccs2024/blob/main/skip-connection.cu"
    click FFN_CUDA "https://github.com/jvhs0706/zkllm-ccs2024/blob/main/ffn.cu"

    %% Click Events for CUDA Kernel Modules: Arithmetic & Tensor Ops
    click ECC "https://github.com/jvhs0706/zkllm-ccs2024/blob/main/bls12-381.cu"
    click FR_Tensor "https://github.com/jvhs0706/zkllm-ccs2024/blob/main/fr-tensor.cu"
    click FR_Tensor "https://github.com/jvhs0706/zkllm-ccs2024/blob/main/fr-tensor.cuh"
    click G1_Tensor "https://github.com/jvhs0706/zkllm-ccs2024/blob/main/g1-tensor.cu"
    click G1_Tensor "https://github.com/jvhs0706/zkllm-ccs2024/blob/main/g1-tensor.cuh"

    %% Click Events for CUDA Kernel Modules: ZK Proof & Specialized Functions
    click Proof "https://github.com/jvhs0706/zkllm-ccs2024/blob/main/proof.cu"
    click Proof "https://github.com/jvhs0706/zkllm-ccs2024/blob/main/proof.cuh"
    click Polynomial "https://github.com/jvhs0706/zkllm-ccs2024/blob/main/polynomial.cu"
    click Polynomial "https://github.com/jvhs0706/zkllm-ccs2024/blob/main/polynomial.cuh"
    click Rescaling "https://github.com/jvhs0706/zkllm-ccs2024/blob/main/rescaling.cu"
    click Rescaling "https://github.com/jvhs0706/zkllm-ccs2024/blob/main/rescaling.cuh"
    click ZKFC "https://github.com/jvhs0706/zkllm-ccs2024/blob/main/zkfc.cu"
    click ZKFC "https://github.com/jvhs0706/zkllm-ccs2024/blob/main/zkfc.cuh"
    click ZKReLU "https://github.com/jvhs0706/zkllm-ccs2024/blob/main/zkrelu.cu"
    click ZKReLU "https://github.com/jvhs0706/zkllm-ccs2024/blob/main/zkrelu.cuh"
    click ZKSoftmax "https://github.com/jvhs0706/zkllm-ccs2024/blob/main/zksoftmax.cu"
    click ZKSoftmax "https://github.com/jvhs0706/zkllm-ccs2024/blob/main/zksoftmax.cuh"
    click TLookup "https://github.com/jvhs0706/zkllm-ccs2024/blob/main/tlookup.cu"
    click TLookup "https://github.com/jvhs0706/zkllm-ccs2024/blob/main/tlookup.cuh"

    %% Click Events for Utility & I/O Components
    click IO_Py "https://github.com/jvhs0706/zkllm-ccs2024/blob/main/fileio_utils.py"
    click IO_CUDA "https://github.com/jvhs0706/zkllm-ccs2024/blob/main/ioutils.cu"

    %% Click Events for Performance Monitoring & Build
    click Timer "https://github.com/jvhs0706/zkllm-ccs2024/blob/main/timer.cpp"
    click Timer "https://github.com/jvhs0706/zkllm-ccs2024/blob/main/timer.hpp"
    click Make "https://github.com/jvhs0706/zkllm-ccs2024/tree/main/Makefile"

    %% Class Definitions for Colors
    classDef orchestration fill:#ADD8E6,stroke:#000,stroke-width:2px;
    classDef cuda fill:#90EE90,stroke:#000,stroke-width:2px;
    classDef io fill:#FFA500,stroke:#000,stroke-width:2px;
    classDef performance fill:#FF6347,stroke:#000,stroke-width:2px;
```
