How jvhs0706/zkllm-ccs2024 works
Project Type & Purpose
• This is a specialized system showing both the cryptographic and deep learning components unified in a single CUDA-based codebase.
• The overall goal is to compute fixed-point representations of model parameters, generate corresponding commitments, and produce layered proofs for neural network/LLM computations.
• The system orchestrates GPU-accelerated computation (via CUDA routines) alongside Python scripts that handle model downloading, public parameter generation, and proof assembly.
Analyzing the File Structure
• There is a mix of CUDA source files (.cu, .cuh) that implement arithmetic over the BLS12-381 curve, tensor operations, and core LLM components (like self-attention, RMSNorm, skip connections, FFN, and polynomial calculations).
• Standalone Python scripts (e.g., download-models.py, llama-ppgen.py, llama-commit.py, llama-ffn.py, etc.) serve as higher-level orchestrators that invoke these CUDA kernels. They deal with tasks such as model downloading, parameter commitment generation, and managing layer proofs.
• Supporting files like Makefile, timer.cpp/ timer.hpp indicate build and timing management for performance benchmarking.
• The naming conventions indicate repeated patterns (e.g., similar CUDA file names for each component such as self-attn, skip-connection, and rmsnorm) which map directly to operational blocks in an LLM layer’s proof computation.
Key Components
a. Front-end / Orchestration Scripts:
• Python scripts: download-models.py (manages model acquisition), llama-ppgen.py (generates public parameters), llama-commit.py (produces fixed-point commitments), and various llama-*.py components that run specific steps for each layer.
b. CUDA Kernels & Components:
• Core computation modules (.cu/.cuh) such as:
• Elliptic curve arithmetic (bls12-381.cu, bls12-381.cuh)
• Tensor operations (e.g., g1-tensor, fr-tensor implementations)
• Proof computations (proof.cu, proof.cuh)
• Specialized functions for LLM layer operations (rmsnorm.cu, self-attn.cu, ffn.cu, skip-connection.cu, ppgen.cu)
• Dedicated modules for other cryptographic primitives (zkfc.cu, zkrelu.cu, zksoftmax.cu, tlookup.cu)
c. Data Management & I/O:
• File input/output utilities (fileio_utils.py, ioutils.cu/cuh) handling the reading/writing of fixed-point tensor data (.bin files).
d. Auxiliary Components:
• Timer utilities built in C++ (timer.cpp, timer.hpp) for performance measurements.
• Build and configuration via Makefile (specifying CUDA architectures like sm_86 for NVIDIA cards).
Relationships and Interactions
• How the public parameter generation (via llama-ppgen.py) is used as an input for the commitment process (llama-commit.py) to convert floating-point to fixed-point values.
• Sequential processing pipeline for each LLM layer:
i. Input preparation and RMSNorm – using llama-rmsnorm.py.
ii. Self-attention computations – using llama-self-attn.py.
iii. Skip connection – using llama-skip-connection.py.
iv. Feed-forward layer (FFN) – using llama-ffn.py.
• External dependencies where required (for instance, torch, transformers, and datasets, even if they’re not heavily integrated with the CUDA code, they provide context regarding the model’s ecosystem).
Architectural Patterns & Design Principles
• The design leverages the separation of concerns by isolating:
• Cryptographic routines (elliptic curve operations, commitment generation).
• Transformer component computations (via modular functions per network block).
• A coupling between numerical precision management (fixed-point conversions) and the corresponding proof generation.
• Python wrapper/orchestration components
• CUDA computational kernels (lower-level arithmetic and tensor operations)
• Data I/O & utility components
Elements to be Included
• All major components should be clearly labeled:
• "Model Downloader" (download-models.py)
• "Public Parameter Generator" (llama-ppgen.py)
• "Commitment Generator" (llama-commit.py)
• "Layer Processing Modules" (each of llama-rmsnorm.py, llama-self-attn.py, llama-skip-connection.py, llama-ffn.py)
• "CUDA Kernel Modules" (group similar CUDA files into blocks like "Elliptic Curve Arithmetic", "Tensor Operations", "ZK Proof Modules")
• "Utility and I/O Components" (fileio_utils.py, ioutils.cu/cuh)
• "Performance Monitoring" (timer.cpp/timer.hpp)
• Data flow: e.g., model parameters flow into “Commitment Generator” then distributed into each layer’s processing.
• Control flow: Python scripts invoking successive CUDA operations.
Happy diagramming!