How flagos-ai/flaggems works
FlagGems is a Triton-based, PyTorch-compatible operator library for accelerating training and inference across accelerator backends. Its principal workflow is a PyTorch user invoking familiar APIs, FlagGems registering implementations with ATen, and the selected operator running a kernel to produce tensor results. Operator families include attention, fused LLM operations, and a broad general-purpose collection; runtime modules provide backend detection and tuning configuration. The tree also shows a substantial offline FlagTune workflow and a C++ dispatcher. Some subsystems were not sampled, so their internal links are omitted rather than inferred.
Main entry points: __init__.py FlagGems API · backend Backend selection · operators.h C++ operators · ops General operators
Components 22
PyTorch integration
- PyTorch APIs
- FlagGems API
src/flag_gems/__init__.py - ATen registration
src/flag_gems/__init__.py
Operator kernels
- General operators
src/flag_gems/ops - Attention kernels
src/flag_gems/ops/attention.py - Flash attention
src/flag_gems/ops/flash_kernel.py - Fused LLM ops
src/flag_gems/fused - MoE routing
src/flag_gems/pt2/moe_routing.py - RoPE kernels
src/flag_gems/fused/rotary_embedding.py
Runtime dispatch
- Backend selection
src/flag_gems/runtime/backend - Device detection
src/flag_gems/runtime/backend/device_finder.py - Runtime config
src/flag_gems/runtime/configs_loader.py - Kernel dispatch
src/flag_gems/utils/libentry.py
FlagTune
- Tuning runtime
src/flag_gems/runtime/flagtune.py - Offline tuning
src/flag_gems/flagtune/offline - Benchmark collection
src/flag_gems/flagtune/offline/collection/scheduler.py - Route training
src/flag_gems/flagtune/offline/train/routed.py
C++ interface
- C++ bindings
cpp/csrc/cstub.cpp - C++ operators
cpp/include/flag_gems/operators.h - C++ RoPE
cpp/lib/rotary_embedding.cpp - C++ embedding
cpp/lib/embedding.cpp
Other components
- PyTorch user
Connections 18
- PyTorch user → PyTorch APIs: invokes
README.md - PyTorch APIs → FlagGems API: uses
README.md - FlagGems API → ATen registration: registers
src/flag_gems/__init__.py - ATen registration → General operators: registers
src/flag_gems/__init__.py - FlagGems API → Backend selection: initializes
src/flag_gems/__init__.py - Backend selection → Device detection: detects
src/flag_gems/runtime/__init__.py - Backend selection → Runtime config: loads
src/flag_gems/runtime/__init__.py - General operators → Kernel dispatch: dispatches via
src/flag_gems/ops/attention.py - Attention kernels → Flash attention: calls
src/flag_gems/ops/attention.py - FlagGems API → Fused LLM ops: exports
src/flag_gems/__init__.py - FlagGems API → MoE routing: exposes
src/flag_gems/pt2/moe_routing.py - MoE routing → Fused LLM ops: uses top-k
src/flag_gems/pt2/moe_routing.py - FlagGems API → Tuning runtime: exposes
src/flag_gems/__init__.py - C++ bindings → C++ operators: binds
cpp/csrc/cstub.cpp - C++ operators → C++ RoPE: declares
cpp/include/flag_gems/operators.h - C++ operators → C++ embedding: declares
cpp/include/flag_gems/operators.h - FlagGems API → RoPE kernels: exports
src/flag_gems/__init__.py - Tuning runtime → Runtime config: uses config
src/flag_gems/runtime/__init__.py