How portainer/kubesolo works
1. KubeSolo is organized as two top-level executables: cmd/kubesolo for the in-cluster appliance and cmd/kubesoloctl for lifecycle management. The former runs the Kubernetes stack; the latter installs, upgrades, resets, uninstalls, and wires kubeconfig for local clients.
2. The core architectural idea is “single-node Kubernetes in one process.” Instead of separate control-plane binaries and etcd, the repo assembles the API server, controller manager, kubelet, container runtime, DNS, storage helpers, and optional extras into one coordinated runtime with a small persistent state directory, defaulting to /var/lib/kubesolo.
3. State storage is replaced by Kine over SQLite. The pkg/kine package is the persistence boundary that substitutes for etcd, while internal/core/pki handles the local PKI and kubeconfig material needed for the API server and admin access. This makes the system self-contained on disk rather than depending on a cluster datastore.
4. Kubernetes control-plane components are implemented as separate packages under pkg/kubernetes/: apiserver, controller, kubelet, kubeproxy, and webhook. Each has its own flags, health checks, service definitions, and executor logic, but they are orchestrated as part of the same local appliance rather than independently deployed services.
5. A notable Kubernetes-specific boundary is the custom webhook path in pkg/kubernetes/webhook, which includes NodeSetter. The README calls this out as the replacement for the full scheduler in a single-node environment; it steers workloads without carrying multi-node scheduling machinery.
6. Container execution is handled by pkg/runtime/containerd plus host-level support in internal/runtime. This includes containerd configuration, image management, plugin setup, and lifecycle management, with crun and containerd called out in the README as the runtime stack. The runtime layer also includes filesystem and network helpers for mounts, IP/masquerade logic, and container health.
7. Networking and service exposure are centralized in the runtime and component layers. CoreDNS is packaged under pkg/components/coredns, and load-balancer behavior is handled so Services can get node-IP-based external endpoints in a single-node context. IPv6 can be disabled globally, which affects both DNS reverse zones and kubelet node-address registration.
8. Storage and workload add-ons are bundled as first-class components under pkg/components/: localpath for local persistent storage, coredns for cluster DNS, portainer for Portainer integration, and d2k for a Docker-compatible API endpoint. These are optional but integrated into the same deployment flow and use Kubernetes manifests, RBAC, namespaces, and services from their package boundaries.
9. Configuration and startup flow are driven by CLI flags and environment variables defined in internal/config/flags, then consumed by internal/cli and internal/core/embedded. The --full, --container-mode, --offline, --d2k, --local-storage, and related flags materially change which subsystems are activated and how much of the upstream Kubernetes default behavior is preserved.
10. internal/cli is the orchestration layer for install-time and lifecycle operations. It contains preflight checks, host detection, downloads, kubeconfig management, service management, process execution, reset/uninstall logic, and command handlers. This is the boundary between “user action on the host” and “cluster appliance bootstrap.”
11. Host integration is handled by internal/system and internal/runtime/service. These packages abstract OS-specific concerns such as mounts, kernel modules, monitoring, and service lifecycles across systemd, openrc, runit, s6, sysvinit, upstart, and foreground/container modes. That abstraction is what lets the same binary run as a native service or inside a container.
12. Offline vs online delivery is a major packaging boundary. internal/core/embedded holds embedded images and architecture-specific bundles for offline use, while the build scripts and Dockerfiles in build/ and the release workflows produce per-architecture artifacts. The online variant fetches images from registries; the offline variant carries images and dependencies in the binary or bundle.
13. The repo also includes installation surfaces outside the binaries: install.sh, install-minimal.sh, uninstall.sh, and kubesolo-service.sh. These are thin wrappers around the same lifecycle logic, mainly for bootstrapping on managed hosts and initializing the selected init system.
14. Test and validation structure mirrors the architecture: unit tests cover config, detection, semver, process, service, and runtime helpers; e2e tests in test/e2e exercise actual workloads, storage, config, controllers, and DNS/load-balancer behavior. That split reflects the project’s two critical contracts: host installation correctness and local Kubernetes compatibility.
15. For an architecture diagram, the key flows are: host/installer inputs into kubesoloctl or kubesolo; flags/config into subsystem selection; embedded or downloaded artifacts into runtime startup; runtime services into the local Kubernetes control loop; and cluster state into SQLite via Kine with kubeconfig exported back to the user for external kubectl access.
Main entry points: main.go kubesolo · containerd containerd · apiserver API server · cli CLI orchestration
Components 21
Executables
Top-level entrypoints for appliance and host lifecycle management.
- kubesolo · Main in-cluster executable that starts the single-node Kubernetes appliance.
cmd/kubesolo/main.go - kubesoloctl · Host-facing executable for install, upgrade, reset, uninstall, and kubeconfig wiring.
cmd/kubesoloctl/main.go
Host control
Installation, detection, service management, and environment integration on the host.
- CLI orchestration · Command handlers and lifecycle orchestration for install-time and maintenance operations.
internal/cli - Service mgr · Abstracts systemd, openrc, runit, s6, sysvinit, upstart, foreground, and container modes.
internal/cli/service - Host system · Host detection, modules, monitoring, and mount integration across operating environments.
internal/system
Core state
Embedded assets, configuration selection, and local PKI/state boundaries.
- Flags config · Flag and environment-driven startup selection that controls which subsystems are enabled.
internal/config/flags - Embedded assets · Offline images, architecture bundles, and embedded delivery content for startup.
internal/core/embedded - PKI state · Local PKI, certificates, and kubeconfig material for API server and admin access.
internal/core/pki
Kubernetes stack
In-cluster control-plane and node services run as one appliance.
- Kine store · Persistence boundary that replaces etcd using Kine over SQLite.
pkg/kine - API server · Kubernetes API server package with flags, RBAC, health, and service wiring.
pkg/kubernetes/apiserver - Controller mgr · Controller manager package with execution, validation, health, and service definitions.
pkg/kubernetes/controller - Kubelet · Local node agent with config, kubeconfig, validation, health, and service wiring.
pkg/kubernetes/kubelet - Kube-proxy · Service networking component for local cluster traffic handling.
pkg/kubernetes/kubeproxy - NodeSetter · Custom webhook that replaces full scheduling with single-node placement logic.
pkg/kubernetes/webhook
Add-ons
Optional but integrated cluster components and service extensions.
- CoreDNS · Cluster DNS component packaged as an integrated manifest-driven add-on.
pkg/components/coredns - Local storage · Local persistent volume provider and storage class for single-node workloads.
pkg/components/localpath - D2K · Docker-compatible API endpoint integrated as an optional component.
pkg/components/d2k - Portainer · Optional Portainer integration packaged with Kubernetes manifests and RBAC.
pkg/components/portainer
Runtime layer
Container runtime, networking, filesystem, and persistence plumbing.
- containerd · Runtime executor for container configuration, images, plugins, namespace, and health.
pkg/runtime/containerd - Net helpers · IP, masquerade, IPv6, and network health helpers for cluster connectivity.
internal/runtime/network - FS helpers · Binary and file helpers used by the runtime layer for mounts and artifacts.
internal/runtime/filesystem
Connections 29
- kubesoloctl → CLI orchestration: drives · Lifecycle command entrypoint delegates into CLI orchestration.
- kubesolo → Flags config: reads · Main appliance process consumes startup flags and environment configuration.
- CLI orchestration → Service mgr: starts/stops · Install and maintenance actions manage services through the abstraction layer.
- CLI orchestration → Host system: detects · CLI uses host detection and OS integration before bootstrap.
- CLI orchestration → Embedded assets: downloads/loads · Installer selects embedded or fetched artifacts for offline and online modes.
- CLI orchestration → PKI state: writes · CLI prepares kubeconfig and local certificate material for users and services.
- Flags config → Embedded assets: selects · Flags determine offline, container-mode, and feature bundle selection.
- Flags config → API server: configures · Configuration controls control-plane startup and preserved upstream behavior.
- Flags config → containerd: configures · Startup flags influence runtime mode and image behavior.
- Embedded assets → containerd: supplies · Embedded binaries and images feed runtime startup.
- PKI state → API server: enables · Certificates and kubeconfig material secure API server access.
- API server → Kine store: persists via · API server state is stored through Kine rather than etcd.
- Controller mgr → API server: reconciles via · Controllers operate against the local API server.
- Kubelet → containerd: executes on · Kubelet launches and manages workloads through the container runtime.
- Kube-proxy → Net helpers: routes · Service networking relies on host and cluster network helpers.
- NodeSetter → Kubelet: places pods · NodeSetter webhook steers workloads in lieu of a full scheduler.
- containerd → FS helpers: uses · Runtime uses filesystem helpers for artifacts and mounts.
- containerd → Net helpers: uses · Runtime uses networking helpers for IP and masquerade behavior.
- CoreDNS → API server: registers with · DNS add-on integrates as a cluster service.
- Local storage → Kine store: stores on · Local storage add-on supports persistent volumes on the node.
- D2K → containerd: exposes · Docker-compatible endpoint fronts the container runtime.
- Portainer → API server: manages via · Portainer add-on connects to the cluster control plane.
- kubesolo → containerd: orchestrates · Appliance binary assembles runtime, control plane, and addons into one process boundary.
- kubesolo → API server: starts · Main appliance process brings up the Kubernetes API server.
- kubesolo → Controller mgr: starts · Main appliance process brings up the controller manager.
- kubesolo → Kubelet: starts · Main appliance process brings up the node agent.
- kubesolo → NodeSetter: starts · Main appliance process brings up the single-node placement webhook.
- kubesolo → CoreDNS: starts · Core cluster DNS is part of the appliance runtime.
- kubesolo → Local storage: starts · Local persistent storage is part of the appliance runtime.