How chazapis/hpktainer works
HPK provides HPC users with a private Kubernetes cluster whose nodes run in rootless “bubbles.” The principal workflow starts with Slurm launching bubbles, then Kubernetes schedules Pods to HPK’s kubelet provider, which prepares pod state and submits jobs that run a pause container and user containers. Separate runtime capabilities cover image handling, volumes, logs/exec, and pod networking. The README documents the overall deployment and networking design; source excerpts show key runtime actions, but do not expose every provider-to-handler call, so those links are documented at subsystem level.
Main entry points: podhandler.go Pod lifecycle · provider.go Virtual kubelet provider · hpk-pause/main.go Pod pause process · hpktainer/main.go Container launcher
Components 27
Cluster control
- K3s cluster
- HPK kubelet
cmd/hpk-kubelet/commands/root/root.go - Virtual kubelet provider
internal/provider/provider.go - Node management
internal/provider/node.go
Pod runtime
- Pod lifecycle
internal/compute/podhandler/podhandler.go - Image handling
internal/compute/image/pull.go - Job submission
internal/compute/runtime/submit.go - Pod pause process
cmd/hpk-pause/main.go - Container launcher
cmd/hpktainer/main.go - Pod volumes
internal/compute/volume - Projected volumes
internal/compute/volume/projected/projected.go
Pod networking
- CNI address allocation
internal/network/cni.go - Calico networking
internal/network/calico.go - TAP forwarder
cmd/hpk-net-daemon/main.go - TAP/socket copying
internal/netutil/copy.go
Runtime support
- Atomic volume writes
internal/compute/volume/util/atomic_writer.go - Pod file paths
internal/compute/endpoint/paths.go - Pod event listener
internal/compute/events/listener.go - Filesystem watching
pkg/filenotify/poller.go - Container logs
pkg/container/logs.go - Safe volume subpaths
pkg/hostutil/subpath.go
Other components
- HPC user
- Slurm job
- kubectl
- Application container
- Bubble BGP peers
- External network
Connections 29
- HPC user → Slurm job: submits job
README.md - Slurm job → K3s cluster: launches bubbles
README.md - kubectl → K3s cluster: issues commands
README.md - K3s cluster → HPK kubelet: hosts kubelet
README.md - HPK kubelet → Virtual kubelet provider: configures provider
- Virtual kubelet provider → Node management: manages nodes
- Virtual kubelet provider → Pod lifecycle: updates pods
- Pod lifecycle → Image handling: handles images
- Pod lifecycle → Pod volumes: prepares volumes
- Pod lifecycle → Job submission: submits jobs
- Job submission → Pod pause process: runs pod process
README.md - Pod pause process → Container launcher: spawns containers
README.md - Container launcher → CNI address allocation: allocates address
cmd/hpktainer/main.go - Container launcher → TAP forwarder: starts forwarder
cmd/hpktainer/main.go - TAP forwarder → TAP/socket copying: forwards TAP traffic
cmd/hpk-net-daemon/main.go - TAP forwarder → TAP forwarder: connects socket pair
README.md - Container launcher → Application container: executes container
cmd/hpktainer/main.go - Pod pause process → Application container: hosts applications
README.md - Calico networking → Bubble BGP peers: routes between bubbles
README.md - Bubble BGP peers → External network: routes traffic outward
README.md - Pod volumes → Projected volumes: supports projected data
- Projected volumes → Atomic volume writes: writes volume files
- Pod lifecycle → Pod file paths: uses pod paths
internal/compute/podhandler/podhandler.go - Virtual kubelet provider → Filesystem watching: watches pod state
internal/provider/provider.go - Pod lifecycle → Filesystem watching: registers watches
internal/compute/podhandler/podhandler.go - Pod pause process → K3s cluster: queries pod API
cmd/hpk-pause/main.go - Virtual kubelet provider → Container logs: serves container logs
- Pod volumes → Safe volume subpaths: secures subpaths
- Pod lifecycle → Pod event listener: reports pod events