How godweiyang/nn-cuda-example works
Main Components
• Custom CUDA Kernel: This is implemented in the file "kernel/add2_kernel.cu" with its associated header "include/add2.h". It represents the core computation logic.
• PyTorch Integration:
• C++ Wrapper: "pytorch/add2_ops.cpp" serves as the glue between the CUDA kernel and PyTorch.
• Python Scripts: "pytorch/time.py" (for runtime benchmarking) and "pytorch/train.py" (for model training).
• Build Scripts: "pytorch/setup.py" and "pytorch/CMakeLists.txt" provide multiple build options.
• TensorFlow Integration:
• C++ Wrapper: "tensorflow/add2_ops.cpp" acts as the equivalent integration layer for TensorFlow.
• Python Scripts: "tensorflow/time.py" (for benchmarking) and "tensorflow/train.py" (for model training).
• Build Scripts: "tensorflow/CMakeLists.txt" for CMake-based compilation.
• Build & Compilation Flows: The project supports multiple ways to compile the CUDA operator (using JIT compilation, setuptools, and CMake).
Relationships and Data Flow
• A central block for the CUDA Kernel, including both the CUDA code and its header. This is the low-level implementation executed on an NVIDIA GPU.
• One branch for PyTorch:
• A “PyTorch Adapter” block (label it with “add2_ops.cpp”).
• From this adapter, include a link to the Python scripts (time and train scripts) that call the custom operator.
• A second branch for TensorFlow:
• A “TensorFlow Adapter” block (label it with “add2_ops.cpp”).
• Attach the Python scripts (time and train) that use this adapter.
• That only the CMake build flow is provided for this branch.
• External dependencies:
• Mention that these adapters integrate with PyTorch and TensorFlow respectively, perhaps by representing these frameworks as separate external components that interact with the adapters.
Architectural Patterns and Design Principles
• Modularity: The separation of the CUDA logic (in the “kernel” and “include” directories) from the framework-specific code (within separate “pytorch” and “tensorflow” directories) emphasizes a modular design.
• Multi-Build Support: Different compilation methods (JIT, Setuptools, CMake) are clearly delineated, so include branching build flows that eventually converge to the integration layers.
• Clear Abstraction Layers: The custom operator is abstracted from the deep learning frameworks by using C++ wrappers, ensuring that both Python and C++ are kept in a clean interface.