kernel/hypeman

1. Overall shape

This repository is a Go-based control plane for running container workloads inside VMs. The main executable is the server in cmd/api, which exposes a JWT-authenticated API and coordinates the lower-level subsystems under lib/. The system is strongly layered: API handlers call domain managers, which in turn talk to hypervisors, storage, networking, guest agents, and host OS integrations.

2. Entry points and top-level processes

The primary runtime is cmd/api/main.go, with dependency injection wiring in cmd/api/wire.go. Other binaries are specialized helpers: cmd/gen-jwt mints tokens, cmd/vz-shim supports macOS virtualization behavior, cmd/uffd-pager backs userfaultfd paging, and cmd/test-prewarm supports test/runtime preparation. The repository is therefore a multi-binary server platform, not a single service.

3. API surface and request flow

cmd/api/api contains the generated and handwritten HTTP handlers for VM lifecycle, images, builds, ingress, volumes, snapshots, health, exec, copy, resources, and standby. The API layer is built from OpenAPI definitions (openapi.yaml, oapi-codegen*.yaml) and uses middleware for auth, tracing, and route resolution under lib/middleware and lib/scopes. Requests flow from HTTP handlers into lib/instances, lib/images, lib/builds, lib/ingress, etc., depending on the resource.

4. Instance lifecycle as the central orchestrator

lib/instances is the core domain package. It coordinates create/start/stop/restart/delete/exec/logs/restore/standby/update operations, applies admission and resource limits, manages state, and glues together networking, volumes, guest memory, snapshots, and hypervisor-specific behavior. If you are drawing the architecture, this is the central hub connecting most subsystems.

5. Hypervisor abstraction and platform split

lib/hypervisor defines the abstraction over multiple VM backends. Concrete implementations exist for Cloud Hypervisor, Firecracker, QEMU, and Apple Virtualization.framework (lib/hypervisor/cloudhypervisor, firecracker, qemu, vz). This is a hard architectural boundary: Linux uses KVM-based backends, while macOS uses vz and a shim/binary entitlement path. VM process management, fork/restore, serial I/O, vsock, and snapshot handling differ by backend.

6. Guest-side communication and control plane

lib/guest defines the guest agent protocol via protobuf/gRPC. lib/system/guest_agent is the code that runs inside the VM guest, while lib/system/init is the guest init layer responsible for early boot setup such as network, mounts, volumes, egress proxy, and Rosetta-related behavior on macOS. The host talks to the guest over vsock and uses these components for exec, file copy, shutdown, and state reporting.

7. Networking, ingress, and external connectivity

lib/network manages host-side bridge allocation, IP derivation, and traffic shaping. lib/ingress provides reverse-proxy management with TLS termination, log handling, and persistent storage for ingress definitions. lib/dns is the embedded DNS server used for routing/subdomain behavior. Together these create the external network boundary for instances, with API-managed ingress resources translating to host network/proxy configuration.

8. Images, OCI cache, and build pipeline

lib/images handles OCI image resolution, storage, disk layout, platform selection, mirrors, and queueing. lib/registry manages registry auth/token flows, while lib/ocicachegc reclaims cache space. lib/builds is a separate subsystem for building images, managing build queues, caching, registry tokens, and a builder agent (lib/builds/builder_agent). The data path here is: pull/build source → OCI metadata/blobs → local cache/storage → VM image materialization.

9. Storage, snapshots, standby, and restore

Persistent VM state is handled across several packages: lib/snapshot, lib/scheduledsnapshots, lib/imageretention, lib/forkvm, and snapshot-related code inside lib/instances and hypervisor packages. Standby/restore is a major feature: a running VM can be snapshotted to disk, the hypervisor stopped, and later resumed. The architecture separates snapshot records/policies from the hypervisor-specific mechanics of saving and restoring memory/device state.

10. Resource management and host feedback loops

lib/resources, lib/vm_metrics, lib/guestmemory, and lib/healthcheck form the resource and health control loop. These packages derive CPU/memory/disk/network/GPU constraints, monitor host and guest pressure, compute balloon targets, and drive health policy decisions. lib/autostandby adds a policy engine that watches connection activity and conntrack events to automatically suspend idle instances.

11. Device and GPU support

lib/devices manages device discovery, VFIO/mdev attachment, GPU modes, and reconciliation. This subsystem is platform-aware and includes tests/data for CUDA-based inference workloads. In the architecture diagram, treat GPU support as an optional hardware acceleration path that plugs into instance admission and hypervisor configuration rather than a core dependency.

12. Operational concerns: auth, observability, and service integration

Authentication is JWT-based, with CLI/API auth helpers in lib/middleware/oapi_auth.go and token generation via cmd/gen-jwt. Observability is built in through lib/otel, tracing helpers in lib/hypervisor, lib/instances, and metrics packages throughout the repo. Deployment artifacts include system service integration for Linux/macOS installers, GitHub Actions, GoReleaser, and AWS CloudFormation under deploy/aws, indicating the server is intended to run as a managed host service rather than a transient CLI tool.