How deeplink-org/dlblas works
Project Type and Purpose
• DLBlas is an open-source, Python-based operator library for Triton kernels.
High-Level Architecture
• API Layer
• Entry points for users:
– dlblas.utils.get_op(…) fetches a tuned operator.
– Direct imports like from dlblas.kernels.rms_norm import rms_norm.
– Top-level calls like dlblas.topk_gating(...).
• Frontend Parsers (dlblas/frontend)
• FX-based parsers that transform user code or Torch modules into an intermediate representation (IR).
• Operator Registry (dlblas/op_registry.py, dlblas/op_struct.py)
• Central registry mapping operator names and signatures to available kernel implementations.
• Autotuner Subsystem (dlblas/autotune)
• Search Space (space.py) defines tunable parameters (block sizes, tile sizes).
• Policy (policy.py) implements search strategies (Bayesian, heuristic).
• Dynamic Compiler & Passes (dynamic_compiler.py, passes.py) generate and optimize kernel code.
• Autotuner (autotuner.py, configs.py) orchestrates the search, measures performance, and picks the best configuration.
• Cache (dlblas/cache.py)
• Kernel Implementations (dlblas/kernels)
• Hardware-specific and Triton kernels organized by backend:
– ascend, camb, cutlass, pure Triton, FP8 variants, fused operations, etc.
• Layer Abstractions (dlblas/layers)
• Higher-level modules, e.g., MoE (dlblas/layers/moe) built atop kernels.
• Utilities (dlblas/utils)
• Device queries, logging, op collection, MOE helpers, general helpers.
• Examples & Benchmarks
• benchmarks/: performance scripts per hardware and kernel.
• examples/: end-to-end training scripts for MoE models.
• Tests (tests/)
• Unit and e2e tests validating kernels, autotuning, caching, and end-to-end scenarios.
Component Relationships and Data Flow
1. User Code calls API Layer.
2. API invokes Frontend Parsers (optional) to build IR.
3. Registry looks up candidate kernels for the requested op/signature.
4. Cache is queried:
• Hit: return cached kernel + config.
• Miss: pass candidates to Autotuner Subsystem.
5. Autotuner runs benchmark loops via Dynamic Compiler and Search Policy over the Search Space, collects performance metrics.
6. Best config is chosen and stored in the Cache.
7. Kernel implementation is loaded from Kernel Modules and executed on device.
8. Utilities provide device info, logging, and metrics throughout.
Architectural Patterns and Principles
• Plugin/Registry Pattern for kernel discovery and extension.
• Strategy Pattern for search policies (swap in Bayesian vs. random).
• Caching Layer to avoid redundant tuning.
• Separation of Concerns: parsing, registry, tuning, caching, execution all modular.
• Extensibility: new hardware backends or kernels drop into dlblas/kernels and auto-register.