# project-hami/hami architecture

> Architecture diagram of the GitHub repository project-hami/hami, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-09-20). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/project-hami/hami
- Repository: https://github.com/project-hami/hami

## Overview

HAMi is Kubernetes GPU virtualization and heterogeneous accelerator scheduling middleware. A workload is submitted to the Kubernetes API, admitted and mutated by the webhook, filtered/scored/bound by the scheduler, then allocated by a device plugin during kubelet startup. The plugin selects physical or virtual devices, emits CDI/runtime configuration, and the workload runs under device-specific limits. Monitoring exports accelerator usage to Prometheus. The graph emphasizes documented workflow and sampled implementations; several vendor backends are represented by their shared device registry rather than unsampled internals.

Main entry points: [`pkg/scheduler/scheduler.go`](https://github.com/project-hami/hami/blob/master/pkg/scheduler/scheduler.go) (Scheduling Core), [`pkg/device-plugin/nvidiadevice/nvinternal/plugin/server.go`](https://github.com/project-hami/hami/blob/master/pkg/device-plugin/nvidiadevice/nvinternal/plugin/server.go) (Device Plugin Server), [`pkg/scheduler/webhook.go`](https://github.com/project-hami/hami/blob/master/pkg/scheduler/webhook.go) (Mutation Webhook), [`cmd/device-plugin/nvidia/plugin-manager.go`](https://github.com/project-hami/hami/blob/master/cmd/device-plugin/nvidia/plugin-manager.go) (Plugin Manager)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_admission_scheduling["Admission &amp; Scheduling"]
  node_hami_webhook["Mutation Webhook<br/>[webhook.go]"]
  node_hami_scheduler_main["Scheduler Service<br/>[main.go]"]
  node_scheduler_routes["Scheduler Routes<br/>[route.go]"]
  node_scheduler_core["Scheduling Core<br/>[scheduler.go]"]
  node_scheduling_policies["Scheduling Policies<br/>[gpu_policy.go]"]
  node_quota_manager["Quota Accounting<br/>[quota.go]"]
end

subgraph group_device_runtime["Device Runtime"]
  node_nvidia_plugin_main["NVIDIA Plugin<br/>[main.go]"]
  node_plugin_manager["Plugin Manager<br/>[plugin-manager.go]"]
  node_plugin_server["Device Plugin Server<br/>[server.go]"]
  node_resource_manager["Resource Manager<br/>[rm.go]"]
  node_cdi_handler["CDI Handler<br/>[cdi.go]"]
end

subgraph group_accelerator_backends["Accelerator Backends"]
  node_device_registry["Device Registry<br/>[devices.go]"]
  node_nvidia_rm["NVIDIA Resource Manager<br/>[nvml_manager.go]"]
  node_tegra_rm["Tegra Resource Manager<br/>[tegra_manager.go]"]
end

subgraph group_observability["Observability"]
  node_v_gpu_monitor["vGPU Monitor<br/>[main.go]"]
  node_metrics_collector["Metrics Collector<br/>[metrics.go]"]
end

node_workload_submitter(("Workload Submitter"))
node_kubernetes_api["Kubernetes API"]
node_kube_scheduler(("Kubernetes Scheduler"))
node_kubelet(("Kubelet"))
node_accelerator_hardware["Accelerator Hardware"]
node_prometheus["Prometheus"]

node_workload_submitter -->|"submits pod"| node_kubernetes_api
node_kubernetes_api -->|"dispatches admission"| node_hami_webhook
node_hami_webhook -->|"mutates resources"| node_device_registry
node_hami_webhook -->|"checks quota"| node_quota_manager
node_kube_scheduler -->|"calls extender"| node_scheduler_routes
node_scheduler_routes -->|"dispatches request"| node_scheduler_core
node_scheduler_core -->|"applies policy"| node_scheduling_policies
node_scheduler_core -->|"fits devices"| node_device_registry
node_scheduler_core -->|"accounts usage"| node_quota_manager
node_scheduler_core -->|"writes annotations"| node_kubernetes_api
node_hami_scheduler_main -->|"starts scheduler"| node_scheduler_core
node_kubernetes_api -->|"delivers pod"| node_kubelet
node_kubelet -->|"starts plugin"| node_nvidia_plugin_main
node_nvidia_plugin_main -->|"creates plugins"| node_plugin_manager
node_plugin_manager -->|"creates CDI"| node_cdi_handler
node_plugin_manager -->|"configures server"| node_plugin_server
node_plugin_server -->|"allocates devices"| node_resource_manager
node_resource_manager -->|"uses NVML"| node_nvidia_rm
node_resource_manager -.->|"selects Tegra backend"| node_tegra_rm
node_nvidia_rm -->|"discovers devices"| node_accelerator_hardware
node_plugin_server -->|"emits runtime config"| node_cdi_handler
node_plugin_server -->|"returns allocation"| node_kubelet
node_kubelet -->|"runs workload"| node_accelerator_hardware
node_v_gpu_monitor -->|"collects usage"| node_metrics_collector
node_metrics_collector -->|"exports metrics"| node_prometheus

click node_hami_webhook "https://github.com/project-hami/hami/blob/master/pkg/scheduler/webhook.go"
click node_hami_scheduler_main "https://github.com/project-hami/hami/blob/master/cmd/scheduler/main.go"
click node_scheduler_routes "https://github.com/project-hami/hami/blob/master/pkg/scheduler/routes/route.go"
click node_scheduler_core "https://github.com/project-hami/hami/blob/master/pkg/scheduler/scheduler.go"
click node_scheduling_policies "https://github.com/project-hami/hami/blob/master/pkg/scheduler/policy/gpu_policy.go"
click node_quota_manager "https://github.com/project-hami/hami/blob/master/pkg/device/quota.go"
click node_nvidia_plugin_main "https://github.com/project-hami/hami/blob/master/cmd/device-plugin/nvidia/main.go"
click node_plugin_manager "https://github.com/project-hami/hami/blob/master/cmd/device-plugin/nvidia/plugin-manager.go"
click node_plugin_server "https://github.com/project-hami/hami/blob/master/pkg/device-plugin/nvidiadevice/nvinternal/plugin/server.go"
click node_resource_manager "https://github.com/project-hami/hami/blob/master/pkg/device-plugin/nvidiadevice/nvinternal/rm/rm.go"
click node_cdi_handler "https://github.com/project-hami/hami/blob/master/pkg/device-plugin/nvidiadevice/nvinternal/cdi/cdi.go"
click node_device_registry "https://github.com/project-hami/hami/blob/master/pkg/device/devices.go"
click node_nvidia_rm "https://github.com/project-hami/hami/blob/master/pkg/device-plugin/nvidiadevice/nvinternal/rm/nvml_manager.go"
click node_tegra_rm "https://github.com/project-hami/hami/blob/master/pkg/device-plugin/nvidiadevice/nvinternal/rm/tegra_manager.go"
click node_v_gpu_monitor "https://github.com/project-hami/hami/blob/master/cmd/vGPUmonitor/main.go"
click node_metrics_collector "https://github.com/project-hami/hami/blob/master/cmd/vGPUmonitor/metrics.go"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_hami_webhook,node_hami_scheduler_main,node_scheduler_routes,node_scheduler_core,node_scheduling_policies,node_quota_manager toneBlue
class node_nvidia_plugin_main,node_plugin_manager,node_plugin_server,node_resource_manager,node_cdi_handler toneAmber
class node_device_registry,node_nvidia_rm,node_tegra_rm,node_kubernetes_api toneMint
class node_v_gpu_monitor,node_metrics_collector,node_kube_scheduler toneRose
class node_workload_submitter,node_kubelet,node_accelerator_hardware,node_prometheus toneIndigo
```

## Components

### Admission & Scheduling

- [Mutation Webhook](https://github.com/project-hami/hami/blob/master/pkg/scheduler/webhook.go): `pkg/scheduler/webhook.go`
- [Scheduler Service](https://github.com/project-hami/hami/blob/master/cmd/scheduler/main.go): `cmd/scheduler/main.go`
- [Scheduler Routes](https://github.com/project-hami/hami/blob/master/pkg/scheduler/routes/route.go): `pkg/scheduler/routes/route.go`
- [Scheduling Core](https://github.com/project-hami/hami/blob/master/pkg/scheduler/scheduler.go): `pkg/scheduler/scheduler.go`
- [Scheduling Policies](https://github.com/project-hami/hami/blob/master/pkg/scheduler/policy/gpu_policy.go): `pkg/scheduler/policy/gpu_policy.go`
- [Quota Accounting](https://github.com/project-hami/hami/blob/master/pkg/device/quota.go): `pkg/device/quota.go`

### Device Runtime

- [NVIDIA Plugin](https://github.com/project-hami/hami/blob/master/cmd/device-plugin/nvidia/main.go): `cmd/device-plugin/nvidia/main.go`
- [Plugin Manager](https://github.com/project-hami/hami/blob/master/cmd/device-plugin/nvidia/plugin-manager.go): `cmd/device-plugin/nvidia/plugin-manager.go`
- [Device Plugin Server](https://github.com/project-hami/hami/blob/master/pkg/device-plugin/nvidiadevice/nvinternal/plugin/server.go): `pkg/device-plugin/nvidiadevice/nvinternal/plugin/server.go`
- [Resource Manager](https://github.com/project-hami/hami/blob/master/pkg/device-plugin/nvidiadevice/nvinternal/rm/rm.go): `pkg/device-plugin/nvidiadevice/nvinternal/rm/rm.go`
- [CDI Handler](https://github.com/project-hami/hami/blob/master/pkg/device-plugin/nvidiadevice/nvinternal/cdi/cdi.go): `pkg/device-plugin/nvidiadevice/nvinternal/cdi/cdi.go`

### Accelerator Backends

- [Device Registry](https://github.com/project-hami/hami/blob/master/pkg/device/devices.go): `pkg/device/devices.go`
- [NVIDIA Resource Manager](https://github.com/project-hami/hami/blob/master/pkg/device-plugin/nvidiadevice/nvinternal/rm/nvml_manager.go): `pkg/device-plugin/nvidiadevice/nvinternal/rm/nvml_manager.go`
- [Tegra Resource Manager](https://github.com/project-hami/hami/blob/master/pkg/device-plugin/nvidiadevice/nvinternal/rm/tegra_manager.go): `pkg/device-plugin/nvidiadevice/nvinternal/rm/tegra_manager.go`

### Observability

- [vGPU Monitor](https://github.com/project-hami/hami/blob/master/cmd/vGPUmonitor/main.go): `cmd/vGPUmonitor/main.go`
- [Metrics Collector](https://github.com/project-hami/hami/blob/master/cmd/vGPUmonitor/metrics.go): `cmd/vGPUmonitor/metrics.go`

### Other components

- **Workload Submitter**
- **Kubernetes API**
- **Kubernetes Scheduler**
- **Kubelet**
- **Accelerator Hardware**
- **Prometheus**

## Connections

- Workload Submitter → Kubernetes API: submits pod
- Kubernetes API → Mutation Webhook: dispatches admission
- Mutation Webhook → Device Registry: mutates resources
- Mutation Webhook → Quota Accounting: checks quota
- Kubernetes Scheduler → Scheduler Routes: calls extender
- Scheduler Routes → Scheduling Core: dispatches request
- Scheduling Core → Scheduling Policies: applies policy
- Scheduling Core → Device Registry: fits devices
- Scheduling Core → Quota Accounting: accounts usage
- Scheduling Core → Kubernetes API: writes annotations
- Scheduler Service → Scheduling Core: starts scheduler
- Kubernetes API → Kubelet: delivers pod
- Kubelet → NVIDIA Plugin: starts plugin
- NVIDIA Plugin → Plugin Manager: creates plugins
- Plugin Manager → CDI Handler: creates CDI
- Plugin Manager → Device Plugin Server: configures server
- Device Plugin Server → Resource Manager: allocates devices
- Resource Manager → NVIDIA Resource Manager: uses NVML
- Resource Manager → Tegra Resource Manager: selects Tegra backend
- NVIDIA Resource Manager → Accelerator Hardware: discovers devices
- Device Plugin Server → CDI Handler: emits runtime config
- Device Plugin Server → Kubelet: returns allocation
- Kubelet → Accelerator Hardware: runs workload
- vGPU Monitor → Metrics Collector: collects usage
- Metrics Collector → Prometheus: exports metrics
