# kernel/hypeman architecture

> Architecture diagram of the GitHub repository kernel/hypeman, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-06-30). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/kernel/hypeman
- Repository: https://github.com/kernel/hypeman

## Overview

1. **Overall shape**
   This repository is a Go-based control plane for running container workloads inside VMs. The main executable is the server in `cmd/api`, which exposes a JWT-authenticated API and coordinates the lower-level subsystems under `lib/`. The system is strongly layered: API handlers call domain managers, which in turn talk to hypervisors, storage, networking, guest agents, and host OS integrations.

2. **Entry points and top-level processes**
   The primary runtime is `cmd/api/main.go`, with dependency injection wiring in `cmd/api/wire.go`. Other binaries are specialized helpers: `cmd/gen-jwt` mints tokens, `cmd/vz-shim` supports macOS virtualization behavior, `cmd/uffd-pager` backs userfaultfd paging, and `cmd/test-prewarm` supports test/runtime preparation. The repository is therefore a multi-binary server platform, not a single service.

3. **API surface and request flow**
   `cmd/api/api` contains the generated and handwritten HTTP handlers for VM lifecycle, images, builds, ingress, volumes, snapshots, health, exec, copy, resources, and standby. The API layer is built from OpenAPI definitions (`openapi.yaml`, `oapi-codegen*.yaml`) and uses middleware for auth, tracing, and route resolution under `lib/middleware` and `lib/scopes`. Requests flow from HTTP handlers into `lib/instances`, `lib/images`, `lib/builds`, `lib/ingress`, etc., depending on the resource.

4. **Instance lifecycle as the central orchestrator**
   `lib/instances` is the core domain package. It coordinates create/start/stop/restart/delete/exec/logs/restore/standby/update operations, applies admission and resource limits, manages state, and glues together networking, volumes, guest memory, snapshots, and hypervisor-specific behavior. If you are drawing the architecture, this is the central hub connecting most subsystems.

5. **Hypervisor abstraction and platform split**
   `lib/hypervisor` defines the abstraction over multiple VM backends. Concrete implementations exist for Cloud Hypervisor, Firecracker, QEMU, and Apple Virtualization.framework (`lib/hypervisor/cloudhypervisor`, `firecracker`, `qemu`, `vz`). This is a hard architectural boundary: Linux uses KVM-based backends, while macOS uses `vz` and a shim/binary entitlement path. VM process management, fork/restore, serial I/O, vsock, and snapshot handling differ by backend.

6. **Guest-side communication and control plane**
   `lib/guest` defines the guest agent protocol via protobuf/gRPC. `lib/system/guest_agent` is the code that runs inside the VM guest, while `lib/system/init` is the guest init layer responsible for early boot setup such as network, mounts, volumes, egress proxy, and Rosetta-related behavior on macOS. The host talks to the guest over vsock and uses these components for exec, file copy, shutdown, and state reporting.

7. **Networking, ingress, and external connectivity**
   `lib/network` manages host-side bridge allocation, IP derivation, and traffic shaping. `lib/ingress` provides reverse-proxy management with TLS termination, log handling, and persistent storage for ingress definitions. `lib/dns` is the embedded DNS server used for routing/subdomain behavior. Together these create the external network boundary for instances, with API-managed ingress resources translating to host network/proxy configuration.

8. **Images, OCI cache, and build pipeline**
   `lib/images` handles OCI image resolution, storage, disk layout, platform selection, mirrors, and queueing. `lib/registry` manages registry auth/token flows, while `lib/ocicachegc` reclaims cache space. `lib/builds` is a separate subsystem for building images, managing build queues, caching, registry tokens, and a builder agent (`lib/builds/builder_agent`). The data path here is: pull/build source → OCI metadata/blobs → local cache/storage → VM image materialization.

9. **Storage, snapshots, standby, and restore**
   Persistent VM state is handled across several packages: `lib/snapshot`, `lib/scheduledsnapshots`, `lib/imageretention`, `lib/forkvm`, and snapshot-related code inside `lib/instances` and hypervisor packages. Standby/restore is a major feature: a running VM can be snapshotted to disk, the hypervisor stopped, and later resumed. The architecture separates snapshot records/policies from the hypervisor-specific mechanics of saving and restoring memory/device state.

10. **Resource management and host feedback loops**
   `lib/resources`, `lib/vm_metrics`, `lib/guestmemory`, and `lib/healthcheck` form the resource and health control loop. These packages derive CPU/memory/disk/network/GPU constraints, monitor host and guest pressure, compute balloon targets, and drive health policy decisions. `lib/autostandby` adds a policy engine that watches connection activity and conntrack events to automatically suspend idle instances.

11. **Device and GPU support**
   `lib/devices` manages device discovery, VFIO/mdev attachment, GPU modes, and reconciliation. This subsystem is platform-aware and includes tests/data for CUDA-based inference workloads. In the architecture diagram, treat GPU support as an optional hardware acceleration path that plugs into instance admission and hypervisor configuration rather than a core dependency.

12. **Operational concerns: auth, observability, and service integration**
   Authentication is JWT-based, with CLI/API auth helpers in `lib/middleware/oapi_auth.go` and token generation via `cmd/gen-jwt`. Observability is built in through `lib/otel`, tracing helpers in `lib/hypervisor`, `lib/instances`, and metrics packages throughout the repo. Deployment artifacts include system service integration for Linux/macOS installers, GitHub Actions, GoReleaser, and AWS CloudFormation under `deploy/aws`, indicating the server is intended to run as a managed host service rather than a transient CLI tool.

Main entry points: [`lib/instances/manager.go`](https://github.com/kernel/hypeman/blob/main/lib/instances/manager.go) (Instances), [`cmd/api/api/api.go`](https://github.com/kernel/hypeman/blob/main/cmd/api/api/api.go) (HTTP handlers), [`cmd/api/wire.go`](https://github.com/kernel/hypeman/blob/main/cmd/api/wire.go) (Wiring), [`lib/hypervisor/hypervisor.go`](https://github.com/kernel/hypeman/blob/main/lib/hypervisor/hypervisor.go) (Hypervisor API)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_grp_cmd["Binaries"]
  node_cmd_api_main["API server<br/>binary<br/>[main.go]"]
  node_cmd_api_wire["Wiring<br/>bootstrap<br/>[wire.go]"]
  node_cmd_vz_shim["VZ shim<br/>macos helper<br/>[main.go]"]
  node_cmd_uffd_pager["UFFD pager<br/>pager binary<br/>[main.go]"]
  node_cmd_gen_jwt["JWT tool<br/>cli<br/>[main.go]"]
end

subgraph group_grp_api["Control plane"]
  node_cmd_api_handlers["HTTP handlers<br/>api handlers<br/>[api.go]"]
  node_lib_middleware["Middleware<br/>auth+otel<br/>[oapi_auth.go]"]
  node_lib_scopes["Scopes<br/>route binding<br/>[scopes.go]"]
end

subgraph group_grp_core["Runtime core"]
  node_lib_instances{{"Instances<br/>orchestrator<br/>[manager.go]"}}
  node_lib_resources["Resources<br/>limits+monitoring<br/>[resource.go]"]
  node_lib_healthcheck["Health policy<br/>health rules<br/>[policy.go]"]
  node_lib_autostandby["Auto-standby<br/>idle policy<br/>[controller.go]"]
  node_lib_vm_metrics["VM metrics<br/>telemetry<br/>[manager.go]"]
end

subgraph group_grp_platform["Platform services"]
  node_lib_hypervisor["Hypervisor API<br/>vm abstraction<br/>[hypervisor.go]"]
  node_lib_images["Images<br/>oci store<br/>[manager.go]"]
  node_lib_builds["Builds<br/>build pipeline<br/>[manager.go]"]
  node_lib_network["Network<br/>[manager.go]"]
  node_lib_ingress["Ingress<br/>proxy manager<br/>[manager.go]"]
  node_lib_dns["DNS<br/>[server.go]"]
  node_lib_guest["Guest protocol<br/>grpc/vsock<br/>[client.go]"]
  node_lib_system_init["Guest init<br/>guest boot<br/>[main.go]"]
  node_lib_system_agent["Guest agent<br/>guest runtime<br/>[main.go]"]
  node_lib_volumes["Volumes<br/>volume store<br/>[manager.go]"]
  node_lib_snapshot["Snapshots<br/>snapshot store<br/>[store.go]"]
  node_lib_devices["Devices<br/>gpu/vfio<br/>[manager.go]"]
  node_lib_registry["Registry<br/>auth tokens<br/>[registry.go]"]
end

node_cmd_api_main -->|"boots"| node_cmd_api_wire
node_cmd_api_wire -->|"uses"| node_lib_middleware
node_cmd_api_wire -->|"routes"| node_lib_scopes
node_cmd_api_wire -->|"injects"| node_lib_instances
node_cmd_api_handlers -->|"orchestrates"| node_lib_instances
node_cmd_api_handlers -->|"manages"| node_lib_images
node_cmd_api_handlers -->|"drives"| node_lib_builds
node_cmd_api_handlers -->|"configures"| node_lib_ingress
node_cmd_api_handlers -->|"attaches"| node_lib_volumes
node_cmd_api_handlers -->|"restores"| node_lib_snapshot
node_cmd_api_handlers -->|"applies"| node_lib_resources
node_cmd_api_handlers -->|"reports"| node_lib_autostandby
node_lib_instances -->|"controls"| node_lib_hypervisor
node_lib_instances -->|"allocates"| node_lib_network
node_lib_instances -->|"mounts"| node_lib_volumes
node_lib_instances -->|"talks to"| node_lib_guest
node_lib_instances -->|"enforces"| node_lib_resources
node_lib_instances -->|"evaluates"| node_lib_healthcheck
node_lib_instances -->|"suspends"| node_lib_autostandby
node_lib_instances -->|"persists"| node_lib_snapshot
node_lib_instances -->|"observes"| node_lib_vm_metrics
node_lib_instances -->|"admits"| node_lib_devices
node_lib_images -->|"auths"| node_lib_registry
node_lib_images -->|"feeds"| node_lib_builds
node_lib_builds -->|"tokenizes"| node_lib_registry
node_lib_builds -->|"agents"| node_lib_guest
node_lib_network -->|"publishes"| node_lib_dns
node_lib_ingress -->|"binds"| node_lib_network
node_lib_guest -->|"executes"| node_lib_system_agent
node_lib_system_init -->|"configures"| node_lib_network
node_lib_system_init -->|"mounts"| node_lib_volumes
node_lib_system_init -->|"egress"| node_lib_ingress
node_lib_hypervisor -->|"macOS path"| node_cmd_vz_shim
node_lib_hypervisor -->|"paging"| node_cmd_uffd_pager
node_lib_hypervisor -->|"restores"| node_lib_snapshot
node_lib_resources -->|"samples"| node_lib_vm_metrics
node_lib_autostandby -->|"consults"| node_lib_vm_metrics
node_cmd_gen_jwt -->|"auths"| node_cmd_api_main

click node_cmd_api_main "https://github.com/kernel/hypeman/blob/main/cmd/api/main.go"
click node_cmd_api_wire "https://github.com/kernel/hypeman/blob/main/cmd/api/wire.go"
click node_cmd_api_handlers "https://github.com/kernel/hypeman/blob/main/cmd/api/api/api.go"
click node_lib_middleware "https://github.com/kernel/hypeman/blob/main/lib/middleware/oapi_auth.go"
click node_lib_scopes "https://github.com/kernel/hypeman/blob/main/lib/scopes/scopes.go"
click node_lib_instances "https://github.com/kernel/hypeman/blob/main/lib/instances/manager.go"
click node_lib_resources "https://github.com/kernel/hypeman/blob/main/lib/resources/resource.go"
click node_lib_healthcheck "https://github.com/kernel/hypeman/blob/main/lib/healthcheck/policy.go"
click node_lib_autostandby "https://github.com/kernel/hypeman/blob/main/lib/autostandby/controller.go"
click node_lib_hypervisor "https://github.com/kernel/hypeman/blob/main/lib/hypervisor/hypervisor.go"
click node_lib_images "https://github.com/kernel/hypeman/blob/main/lib/images/manager.go"
click node_lib_builds "https://github.com/kernel/hypeman/blob/main/lib/builds/manager.go"
click node_lib_network "https://github.com/kernel/hypeman/blob/main/lib/network/manager.go"
click node_lib_ingress "https://github.com/kernel/hypeman/blob/main/lib/ingress/manager.go"
click node_lib_dns "https://github.com/kernel/hypeman/blob/main/lib/dns/server.go"
click node_lib_guest "https://github.com/kernel/hypeman/blob/main/lib/guest/client.go"
click node_lib_system_init "https://github.com/kernel/hypeman/blob/main/lib/system/init/main.go"
click node_lib_system_agent "https://github.com/kernel/hypeman/blob/main/lib/system/guest_agent/main.go"
click node_lib_volumes "https://github.com/kernel/hypeman/blob/main/lib/volumes/manager.go"
click node_lib_snapshot "https://github.com/kernel/hypeman/blob/main/lib/snapshot/store.go"
click node_lib_vm_metrics "https://github.com/kernel/hypeman/blob/main/lib/vm_metrics/manager.go"
click node_lib_devices "https://github.com/kernel/hypeman/blob/main/lib/devices/manager.go"
click node_lib_registry "https://github.com/kernel/hypeman/blob/main/lib/registry/registry.go"
click node_cmd_vz_shim "https://github.com/kernel/hypeman/blob/main/cmd/vz-shim/main.go"
click node_cmd_uffd_pager "https://github.com/kernel/hypeman/blob/main/cmd/uffd-pager/main.go"
click node_cmd_gen_jwt "https://github.com/kernel/hypeman/blob/main/cmd/gen-jwt/main.go"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_cmd_api_main,node_cmd_api_wire,node_cmd_vz_shim,node_cmd_uffd_pager,node_cmd_gen_jwt toneBlue
class node_cmd_api_handlers,node_lib_middleware,node_lib_scopes toneAmber
class node_lib_instances,node_lib_resources,node_lib_healthcheck,node_lib_autostandby,node_lib_vm_metrics toneMint
class node_lib_hypervisor,node_lib_images,node_lib_builds,node_lib_network,node_lib_ingress,node_lib_dns,node_lib_guest,node_lib_system_init,node_lib_system_agent,node_lib_volumes,node_lib_snapshot,node_lib_devices,node_lib_registry toneRose
```

## Components

### Binaries

Executable entry points and helper runtimes.

- [API server](https://github.com/kernel/hypeman/blob/main/cmd/api/main.go): binary · Main control-plane process exposing the authenticated VM API. · `cmd/api/main.go`
- [Wiring](https://github.com/kernel/hypeman/blob/main/cmd/api/wire.go): bootstrap · Dependency injection and service assembly for the API server. · `cmd/api/wire.go`
- [VZ shim](https://github.com/kernel/hypeman/blob/main/cmd/vz-shim/main.go): macos helper · macOS Virtualization.framework helper for Rosetta and save/restore behavior. · `cmd/vz-shim/main.go`
- [UFFD pager](https://github.com/kernel/hypeman/blob/main/cmd/uffd-pager/main.go): pager binary · Userfaultfd paging helper used by the VM runtime. · `cmd/uffd-pager/main.go`
- [JWT tool](https://github.com/kernel/hypeman/blob/main/cmd/gen-jwt/main.go): cli · Token generator for API authentication. · `cmd/gen-jwt/main.go`

### Control plane

HTTP API, auth, routing, and request handling.

- [HTTP handlers](https://github.com/kernel/hypeman/blob/main/cmd/api/api/api.go): api handlers · Lifecycle, images, builds, ingress, volumes, snapshots, health, exec, copy, and standby endpoints. · `cmd/api/api/api.go`
- [Middleware](https://github.com/kernel/hypeman/blob/main/lib/middleware/oapi_auth.go): auth+otel · JWT auth, OpenTelemetry, and request resolution middleware. · `lib/middleware/oapi_auth.go`
- [Scopes](https://github.com/kernel/hypeman/blob/main/lib/scopes/scopes.go): route binding · Route-to-resource scoping and middleware integration. · `lib/scopes/scopes.go`

### Runtime core

Instance orchestration and its major subsystems.

- [Instances](https://github.com/kernel/hypeman/blob/main/lib/instances/manager.go): orchestrator · Central VM lifecycle coordinator linking policy, storage, networking, guest control, and hypervisors. · `lib/instances/manager.go`
- [Resources](https://github.com/kernel/hypeman/blob/main/lib/resources/resource.go): limits+monitoring · CPU, memory, disk, network, and GPU constraint evaluation and feedback loops. · `lib/resources/resource.go`
- [Health policy](https://github.com/kernel/hypeman/blob/main/lib/healthcheck/policy.go): health rules · Instance health evaluation and probing policy. · `lib/healthcheck/policy.go`
- [Auto-standby](https://github.com/kernel/hypeman/blob/main/lib/autostandby/controller.go): idle policy · Activity-driven suspend policy using conntrack and runtime signals. · `lib/autostandby/controller.go`
- [VM metrics](https://github.com/kernel/hypeman/blob/main/lib/vm_metrics/manager.go): telemetry · Guest and host VM metric collection and aggregation. · `lib/vm_metrics/manager.go`

### Platform services

Guest, hypervisor, storage, and host integrations.

- [Hypervisor API](https://github.com/kernel/hypeman/blob/main/lib/hypervisor/hypervisor.go): vm abstraction · Common VM backend interface shared by Cloud Hypervisor, Firecracker, QEMU, and VZ. · `lib/hypervisor/hypervisor.go`
- [Images](https://github.com/kernel/hypeman/blob/main/lib/images/manager.go): oci store · OCI image resolution, caching, storage, and platform selection. · `lib/images/manager.go`
- [Builds](https://github.com/kernel/hypeman/blob/main/lib/builds/manager.go): build pipeline · Image build queue, cache, registry token handling, and builder agent coordination. · `lib/builds/manager.go`
- [Network](https://github.com/kernel/hypeman/blob/main/lib/network/manager.go): host networking · Bridge allocation, addressing, and traffic shaping for instances. · `lib/network/manager.go`
- [Ingress](https://github.com/kernel/hypeman/blob/main/lib/ingress/manager.go): proxy manager · Reverse-proxy and TLS ingress management with persistent definitions. · `lib/ingress/manager.go`
- [DNS](https://github.com/kernel/hypeman/blob/main/lib/dns/server.go): embedded dns · Internal DNS server for instance routing and subdomain behavior. · `lib/dns/server.go`
- [Guest protocol](https://github.com/kernel/hypeman/blob/main/lib/guest/client.go): grpc/vsock · Host-side guest-agent protocol used for exec, copy, shutdown, and status. · `lib/guest/client.go`
- [Guest init](https://github.com/kernel/hypeman/blob/main/lib/system/init/main.go): guest boot · Early boot layer configuring mounts, network, volumes, egress proxy, and Rosetta behavior. · `lib/system/init/main.go`
- [Guest agent](https://github.com/kernel/hypeman/blob/main/lib/system/guest_agent/main.go): guest runtime · In-VM agent implementing exec, copy, network, stat, and shutdown operations. · `lib/system/guest_agent/main.go`
- [Volumes](https://github.com/kernel/hypeman/blob/main/lib/volumes/manager.go): volume store · Volume archive, storage, and lifecycle management. · `lib/volumes/manager.go`
- [Snapshots](https://github.com/kernel/hypeman/blob/main/lib/snapshot/store.go): snapshot store · Snapshot records, policies, and persistence for standby/restore. · `lib/snapshot/store.go`
- [Devices](https://github.com/kernel/hypeman/blob/main/lib/devices/manager.go): gpu/vfio · Device discovery, VFIO/mdev attachment, and GPU reconciliation. · `lib/devices/manager.go`
- [Registry](https://github.com/kernel/hypeman/blob/main/lib/registry/registry.go): auth tokens · Container registry auth and token flows. · `lib/registry/registry.go`

## Connections

- API server → Wiring: boots · Assembles dependencies and starts the control plane.
- Wiring → Middleware: uses · Installs authentication and observability middleware.
- Wiring → Scopes: routes · Binds API endpoints to scoped resources.
- Wiring → Instances: injects · Provides the core instance manager to handlers.
- HTTP handlers → Instances: orchestrates · Lifecycle and control endpoints delegate to instance operations.
- HTTP handlers → Images: manages · Image-related endpoints call the image subsystem.
- HTTP handlers → Builds: drives · Build endpoints enqueue and monitor builds.
- HTTP handlers → Ingress: configures · Ingress resources map to proxy management.
- HTTP handlers → Volumes: attaches · Volume endpoints manage persistent storage.
- HTTP handlers → Snapshots: restores · Snapshot endpoints persist and resume VM state.
- HTTP handlers → Resources: applies · Resource endpoints trigger admission and reclaim actions.
- HTTP handlers → Auto-standby: reports · Auto-standby status endpoints expose policy state.
- Instances → Hypervisor API: controls · Creates, starts, stops, restores, and snapshots VMs through backend abstractions.
- Instances → Network: allocates · Assigns bridges, IPs, and traffic shaping for instances.
- Instances → Volumes: mounts · Connects instance storage and volume archives.
- Instances → Guest protocol: talks to · Uses vsock/gRPC to reach the guest agent.
- Instances → Resources: enforces · Checks resource admission and computes limits.
- Instances → Health policy: evaluates · Uses health policy to decide instance status and restarts.
- Instances → Auto-standby: suspends · Integrates idle-policy decisions into instance standby.
- Instances → Snapshots: persists · Coordinates snapshot creation and restore flows.
- Instances → VM metrics: observes · Feeds runtime telemetry into control decisions.
- Instances → Devices: admits · Optional device and GPU configuration is part of instance setup.
- Images → Registry: auths · Acquires registry credentials and tokens.
- Images → Builds: feeds · Builds consume and materialize image artifacts.
- Builds → Registry: tokenizes · Builds use registry tokens for push/pull access.
- Builds → Guest protocol: agents · Builder agent communicates over guest channels.
- Network → DNS: publishes · DNS routes are coupled to host network allocation.
- Ingress → Network: binds · Ingress endpoints depend on host networking.
- Guest protocol → Guest agent: executes · Guest protocol is implemented by the in-VM agent.
- Guest init → Network: configures · Guest init sets up networking during boot.
- Guest init → Volumes: mounts · Guest init mounts attached volumes.
- Guest init → Ingress: egress · Guest init configures egress proxy behavior.
- Hypervisor API → VZ shim: macOS path · VZ backend uses the shim for platform-specific behavior.
- Hypervisor API → UFFD pager: paging · Some backends rely on the UFFD pager helper.
- Hypervisor API → Snapshots: restores · Backend implementations persist and restore VM state.
- Resources → VM metrics: samples · Uses telemetry to tune allocations and pressure responses.
- Auto-standby → VM metrics: consults · Idle policy incorporates runtime metrics and activity signals.
- JWT tool → API server: auths · Generates JWTs used by the API server.