# hidet-org/hidet architecture

> Architecture diagram of the GitHub repository hidet-org/hidet, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2024-12-28). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/hidet-org/hidet
- Repository: https://github.com/hidet-org/hidet

## Overview

##### Main Components

a) Frontend Layer
- PyTorch Frontend (python/hidet/graph/frontend/torch)
- ONNX Frontend (python/hidet/graph/frontend/onnx)
- Direct Hidet API (python/hidet/graph)

b) IR (Intermediate Representation) Layer (python/hidet/ir)
- Graph IR
- Compute IR
- Schedule IR
- Cute IR (for GPU optimization)

c) Optimization Layer (python/hidet/transforms)
- Graph-level optimizations
- Operator-level optimizations
- Platform-specific optimizations

d) Backend Layer (python/hidet/backend)
- CUDA code generation
- CPU code generation
- Runtime support

e) Runtime System (python/hidet/runtime)
- Memory management
- Device management
- Execution engine

##### Relationships and Data Flow

**Input Models (PyTorch/ONNX) → Frontend Layer**
- ↓
**Graph IR → Optimization Layer**
- ↓
**Compute/Schedule IR → Backend Layer**
- ↓
**Generated Code → Runtime System**
- ↓
**Optimized Execution**

##### Key Subsystems to Highlight

a) Compilation Pipeline:
- Model ingestion
- IR transformation
- Code generation
- Dynamic compilation

b) Optimization Framework:
- Task mapping
- Tensor program optimization
- Memory planning
- Kernel fusion

c) Runtime Components:
- CUDA runtime integration
- Memory management
- Multi-device support
- Caching system

##### Additional Details

a) Support Systems:
- Cache management (python/hidet/utils/cache_utils.py)
- Profiling and debugging tools
- Configuration management

b) Integration Points:
- CUDA/cuBLAS integration
- PyTorch integration
- ONNX runtime integration

##### Optimization Pipeline Details

Optimization flow:
- Graph-level optimizations
- Operator fusion
- Memory planning
- Platform-specific optimizations

## Diagram (Mermaid)

```mermaid
graph TB
    %% Define styles
    classDef frontend fill:#b3d9ff,stroke:#333
    classDef ir fill:#b3ffb3,stroke:#333
    classDef optimization fill:#ffcc99,stroke:#333
    classDef backend fill:#ffb3ff,stroke:#333
    classDef runtime fill:#e6ccff,stroke:#333
    classDef support fill:#f2f2f2,stroke:#333

    subgraph Frontend
        PyTorch["PyTorch Frontend"]:::frontend
        ONNX["ONNX Frontend"]:::frontend
        HidetAPI["Direct Hidet API"]:::frontend
    end

    subgraph IRLayer
        GraphIR["Graph IR"]:::ir
        ComputeIR["Compute IR"]:::ir
        CuteIR["Cute IR"]:::ir
    end

    subgraph Optimization
        GraphOpt["Graph-level Optimizations"]:::optimization
        PlatformOpt["Platform-specific Optimizations"]:::optimization
        GeneralOpt["General Optimizations"]:::optimization
    end

    subgraph Backend
        CUDA["CUDA Backend"]:::backend
        RuntimeSupport["Runtime Support"]:::backend
        CUDAIntegration["CUDA Integration"]:::backend
    end

    subgraph Runtime
        MemoryMgmt["Memory Management"]:::runtime
        DeviceMgmt["Device Management"]:::runtime
        ExecEngine["Execution Engine"]:::runtime
    end

    subgraph Support
        CacheMgmt["Cache Management"]:::support
        CUDALibs["CUDA Libraries Integration"]:::support
    end

    %% Connections
    PyTorch & ONNX & HidetAPI --> GraphIR
    GraphIR --> ComputeIR
    ComputeIR --> CuteIR
    
    GraphIR --> GraphOpt
    CuteIR --> PlatformOpt
    GraphOpt & PlatformOpt --> GeneralOpt
    
    GeneralOpt --> CUDA
    CUDA --> RuntimeSupport
    CUDAIntegration --> RuntimeSupport
    
    RuntimeSupport --> MemoryMgmt
    RuntimeSupport --> DeviceMgmt
    RuntimeSupport --> ExecEngine
    
    CacheMgmt --> ExecEngine
    CUDALibs --> CUDAIntegration

    %% Click events
    click PyTorch "https://github.com/hidet-org/hidet/tree/main/python/hidet/graph/frontend/torch/"
    click ONNX "https://github.com/hidet-org/hidet/tree/main/python/hidet/graph/frontend/onnx/"
    click HidetAPI "https://github.com/hidet-org/hidet/tree/main/python/hidet/graph/"
    click GraphIR "https://github.com/hidet-org/hidet/tree/main/python/hidet/ir/"
    click ComputeIR "https://github.com/hidet-org/hidet/tree/main/python/hidet/ir/compute/"
    click CuteIR "https://github.com/hidet-org/hidet/tree/main/python/hidet/ir/cute/"
    click GraphOpt "https://github.com/hidet-org/hidet/tree/main/python/hidet/graph/transforms/"
    click PlatformOpt "https://github.com/hidet-org/hidet/tree/main/python/hidet/transforms/cute/cuda/"
    click GeneralOpt "https://github.com/hidet-org/hidet/tree/main/python/hidet/transforms/"
    click CUDA "https://github.com/hidet-org/hidet/tree/main/python/hidet/backend/"
    click RuntimeSupport "https://github.com/hidet-org/hidet/tree/main/src/hidet/runtime/"
    click CUDAIntegration "https://github.com/hidet-org/hidet/tree/main/python/hidet/cuda/"
    click MemoryMgmt "https://github.com/hidet-org/hidet/blob/main/python/hidet/runtime/storage.py"
    click DeviceMgmt "https://github.com/hidet-org/hidet/blob/main/python/hidet/runtime/device.py"
    click ExecEngine "https://github.com/hidet-org/hidet/blob/main/python/hidet/runtime/compiled_module.py"
    click CacheMgmt "https://github.com/hidet-org/hidet/blob/main/python/hidet/utils/cache_utils.py"
    click CUDALibs "https://github.com/hidet-org/hidet/tree/main/python/hidet/cuda/cublas/"

    %% Legend
    subgraph Legend
        Frontend_L["Frontend Components"]:::frontend
        IR_L["IR Components"]:::ir
        Opt_L["Optimization Components"]:::optimization
        Backend_L["Backend Components"]:::backend
        Runtime_L["Runtime Components"]:::runtime
        Support_L["Support Systems"]:::support
    end
```
