How kiaghods/butterfly_matrix_kernel works
Step 1: Identify Project Type and Purpose
• This is an open-source, high-performance numerical library/tool focused on butterfly matrix multiplication and related FFT kernels on NVIDIA GPUs.
• It provides Triton GPU kernels, a PyTorch nn.Module wrapper, reference implementations, and a full suite of testing, profiling, benchmarking and optimization scripts.
Step 2: Analyze File Structure
Top-level components:
Core computation
• triton_kernel.py – main butterfly GPU kernel
• triton_fft_kernel.py – discrete FFT using butterfly kernels
• reference_impl.py – pure‐PyTorch reference for correctness
Python module interface
• butterfly_module.py – PyTorch nn.Module wrapper (ButterflyLayer)
Testing
• unittests/ – pytest unit tests (correctness, numwarp tuning)
• testing/ – standalone correctness harness
Profiling
• profiling/torch_profiler.py – PyTorch profiler integration
• profiling/memory_profiler.py – memory usage profiling
• profiling/visualization.py – SQLite-based Nsight Systems visualization
Benchmarking
• benchmarking/performance_bench.py – performance comparison script
Optimization
• optimization/numwarp_tuning.py – warp‐count tuning
• optimization/fused_kernel.py – L=8 fused stage kernel
• optimization/pipelined_kernel.py– pipelined stage kernel
Automation scripts
• scripts/run_profiling.py
• scripts/run_benchmarks.py
Results & artifacts
• results/ – stored profiles, benchmarks, plots, optimization data
Step 3: README Insights
• The README clearly describes each component’s role, usage examples, and the optimization strategies (kernel fusion, pipelining, num_warps).
• It details the FFT implementation as a special use-case of butterfly matmul.
1. User/API Layer
– ButterflyLayer (PyTorch nn.Module)
– FFT wrapper
2. Core GPU Engines
– Triton butterfly kernel
– Triton FFT butterfly stage kernel
3. Reference Implementation
– PyTorch‐only matmul in reference_impl.py
4. Testing Suite
– pytest (unittests/)
– correctness_harness (testing/)
5. Profiling Tools
– TorchProfiler integration
– MemoryProfiler
– Nsight Systems → visualization
6. Benchmarking Suite
– performance_bench.py
7. Optimization Scripts
– numwarp tuning
– pipelining kernel
– fused kernel (L=8)
8. Automation & CI Scripts
– run_profiling.py
– run_benchmarks.py
9. Results Storage
– results/profiles, benchmarks, plots, optimization outputs
– User/API Layer → Core GPU Engines (runtime calls)
– Core GPU Engines ↔ Reference Implementation (correctness validation)
– Testing Suite → API Layer & Core Engines (executes both)
– Profiling Tools ⟶ Core Engines & API Layer (collect metrics)
– Benchmarking Suite ⟶ Core Engines & Reference Impl (performance comparison)
– Optimization Scripts ⟶ Core Engines (generate and test variants)
– Automation Scripts orchestrate Testing, Profiling, Benchmarking
– Results Storage ← all tools (profiling, benchmarking, optimization)
c. Architectural Patterns & Principles
– Layered design: API wrapper, core compute, utilities
– Separation of concerns: computation vs. testing vs. profiling vs. benchmarking
– Data flow: from input tensors → kernels → outputs, then into testing/profiling/benchmarking tools
d. Key Technologies & Frameworks
– Triton (GPU kernels)
– PyTorch (tensor ops, autograd, nn.Module)
– pytest (unit tests)
– Nsight Systems / PyTorch profiler / memory_profiler
– Python scripting for orchestration
• core functionality: GPU kernels and how they integrate with PyTorch.
• extensibility points: reference vs. optimized kernels, custom tuning scripts.
• integration: how profiling and benchmarking plug into the workflow.
Step 6: Diagramming Guidelines
• Use clear, descriptive labels on each node (e.g., “Triton Butterfly Kernel”, “ButterflyLayer nn.Module”).
• Green for core compute engines
• Blue for orchestration/scripts
• Orange for testing/profiling/benchmarking
• Gray for results storage
• Rectangles for code modules
• Parallelograms for user/API interfaces
Step 7: Detailed Capture of Essential Architectural Elements
• Layer (API, compute, validation, instrumentation, storage) is represented.
• Interactions with brief notes (e.g., “compare performance”, “validate output”).
• If space permits, include version or dependencies (e.g., “requires CUDA compute capability ≥7.0”, “Python 3.10+”).