pearblossom-m/flash-swiglu-mlp

Flash SwiGLU MLP is a GPU-optimized PyTorch/Triton library for the SwiGLU feed-forward block used in LLMs. Callers use either the functional API or a weight-owning module; both enter the same autograd operation. Forward fuses the gate and up projections with SiLU and gating, then uses PyTorch linear for the down projection. Training saves intermediates and computes gradients through fused backward kernels and matrix multiplies; inference skips saving gate/up activations. The README also documents Falcon H1 scaling and shape-bucketed autotuning. Some orchestration source was omitted, so the graph leaves uncertain internal dispatch details out.