How hidet-org/hidet works
Main Components
a) Frontend Layer
• PyTorch Frontend (python/hidet/graph/frontend/torch)
• ONNX Frontend (python/hidet/graph/frontend/onnx)
• Direct Hidet API (python/hidet/graph)
b) IR (Intermediate Representation) Layer (python/hidet/ir)
• Graph IR
• Compute IR
• Schedule IR
• Cute IR (for GPU optimization)
c) Optimization Layer (python/hidet/transforms)
• Graph-level optimizations
• Operator-level optimizations
• Platform-specific optimizations
d) Backend Layer (python/hidet/backend)
• CUDA code generation
• CPU code generation
• Runtime support
e) Runtime System (python/hidet/runtime)
• Memory management
• Device management
• Execution engine
Relationships and Data Flow
Input Models (PyTorch/ONNX) → Frontend Layer
• ↓
Graph IR → Optimization Layer
• ↓
Compute/Schedule IR → Backend Layer
• ↓
Generated Code → Runtime System
• ↓
Optimized Execution
Key Subsystems to Highlight
a) Compilation Pipeline:
• Model ingestion
• IR transformation
• Code generation
• Dynamic compilation
b) Optimization Framework:
• Task mapping
• Tensor program optimization
• Memory planning
• Kernel fusion
c) Runtime Components:
• CUDA runtime integration
• Memory management
• Multi-device support
• Caching system
Additional Details
a) Support Systems:
• Cache management (python/hidet/utils/cache_utils.py)
• Profiling and debugging tools
• Configuration management
b) Integration Points:
• CUDA/cuBLAS integration
• PyTorch integration
• ONNX runtime integration
Optimization Pipeline Details
Optimization flow:
• Graph-level optimizations
• Operator fusion
• Memory planning
• Platform-specific optimizations