How karimfin/turb-kub works
1. What this repository is
- This repo packages a custom Kubernetes scheduler written in Python and a small set of manifests to run it in a local Minikube cluster.
- The architecture is centered on the scheduler process watching the Kubernetes API and making placement decisions for incoming pods.
2. Core runtime component
- The main executable is scheduler/scheduler.py.
- It runs as a containerized in-cluster controller rather than as an external tool, so it has to authenticate to the cluster like any other pod.
3. Cluster-facing deployment boundary
- manifests/scheduler-deployment.yaml deploys the scheduler into the cluster and includes the RBAC objects it needs.
- That manifest defines the scheduler’s ServiceAccount, ClusterRole, and ClusterRoleBinding, which is the key security boundary allowing it to read cluster state and bind pods.
4. Scheduler input flow
- The scheduler reacts to pending pod events from the Kubernetes API.
- For each pod to place, it queries cluster state to determine candidate nodes and current workload, using the pod’s requested memory as the scheduling signal.
5. Node selection logic
- The algorithm is least-loaded-first based on memory requests, not actual runtime memory usage.
- It:
- lists Ready nodes,
- totals requested memory of already scheduled pods per node,
- excludes nodes that would exceed a synthetic capacity limit,
- selects the node with the lowest requested memory.
6. Capacity model
- Instead of relying on node allocatable memory, the scheduler uses the NODE_MEM_LIMIT_MB environment variable as a synthetic per-node ceiling.
- This is an architecture choice tied to the Minikube/Docker environment, where node allocatable memory can be unreliable or awkward to use.
7. Placement side effect
- Once a node is chosen, the scheduler performs a bind operation through the Kubernetes API to assign the pod to that node.
- This is the actual “write” action in the system; everything else is read-only cluster inspection.
8. Operational dependencies
- The scheduler depends on Kubernetes API access plus permission to write events so its logs stay clean.
- It runs in the kube-system namespace, but the memory accounting logic shown in the README only considers pods from the default namespace, which is an important architectural constraint.
9. Test workload
- manifests/pod1.yaml, pod2.yaml, and pod3.yaml are example workloads used to validate placement behavior.
- These pods are intentionally sized so the least-loaded policy distributes them across the two Minikube nodes while respecting the 2 GB synthetic cap.
10. Container build boundary
- scheduler/Dockerfile packages the Python scheduler for in-cluster execution.
- The README notes imagePullPolicy: Never, so the image is expected to be built directly into Minikube’s Docker daemon rather than pulled from a registry.
11. Local orchestration and developer workflow
- Makefile is the convenience layer for starting the cluster, building the image, deploying the scheduler, and cleaning up.
- This repo is therefore optimized for a local cluster test loop, not for production deployment automation.
12. Important architectural tradeoffs
- The scheduler is intentionally simple and state-light: it recalculates node load from the API on each scheduling event instead of maintaining an internal cache.
- That keeps the design small and easy to reason about, but it also means higher API usage, limited namespace scope, and no robust retry/reconciliation path for bind failures.
13. How to draw the architecture
- The main boxes are: Minikube cluster, scheduler pod, Kubernetes API server, worker nodes, and test pods.
- Data flow is: test pod manifests are applied → scheduler watches pending pods → scheduler reads nodes and existing pods → scheduler picks a node → scheduler binds the pod → Kubernetes schedules the pod onto the chosen node.
Main entry points: Makefile Makefile · scheduler.py Scheduler Core · scheduler-deployment.yaml Scheduler Deploy · Dockerfile Scheduler Image
Components 16
Local Dev
Build and apply the scheduler stack from the repository root.
- Makefile · Convenience entrypoint for building, deploying, and cleaning the local test loop.
Makefile - Scheduler Image · Builds the container image used to run the scheduler in-cluster.
scheduler/Dockerfile - Scheduler Deploy · Deploys the scheduler pod and defines its RBAC boundary.
manifests/scheduler-deployment.yaml
Cluster Runtime
In-cluster scheduler, API access, and placement flow inside Minikube.
- Scheduler Core · Main scheduling loop that watches pods, scores nodes, and binds placements.
scheduler/scheduler.py - K8s API · Cluster API used to list pods and nodes and to perform bind operations.
- RBAC Boundary · ServiceAccount, ClusterRole, and ClusterRoleBinding required for read and bind access.
- Scheduler Pod · Containerized scheduler process running in kube-system.
- Pod Watch · Watches for pending pods that need placement.
- Node Score · Totals requested memory on ready nodes and ranks them least-loaded-first.
- Capacity Gate · Filters nodes that would exceed the NODE_MEM_LIMIT_MB ceiling.
- Bind Pod · Binds the chosen pod to the selected node through the Kubernetes API.
- Worker Nodes · Ready nodes considered as placement targets by the scheduler.
- Default Pods · Existing pods in the default namespace used for requested-memory accounting.
Workloads
Example pods used to exercise the scheduling policy.
Connections 22
- Makefile → Scheduler Image: build image · Builds the scheduler container directly into the local Minikube Docker daemon.
- Makefile → Scheduler Deploy: deploy · Applies the scheduler deployment and its RBAC objects.
- Makefile → Pod 1: apply · Creates the first example workload.
- Makefile → Pod 2: apply · Creates the second example workload.
- Makefile → Pod 3: apply · Creates the third example workload.
- Scheduler Image → Scheduler Core: packages · Container image bundles the Python scheduler process.
- Scheduler Deploy → RBAC Boundary: defines · Manifest includes the service account and cluster permissions.
- Scheduler Deploy → Scheduler Pod: runs as · Deploys the scheduler into the cluster as a pod.
- Scheduler Pod → Scheduler Core: executes · Runs the scheduler control loop inside the container.
- Scheduler Core → Pod Watch: starts · Begins watching for pending pods.
- Pod Watch → K8s API: reads · Uses the API server to observe pending pods and cluster state.
- K8s API → Worker Nodes: lists · Provides ready node information for placement decisions.
- K8s API → Default Pods: lists · Provides existing default-namespace pods for memory accounting.
- Scheduler Core → Node Score: scores · Computes least-loaded node selection from requested memory.
- Node Score → Capacity Gate: filters · Rejects nodes that would exceed the synthetic memory ceiling.
- Capacity Gate → Worker Nodes: selects from · Chooses among ready nodes that remain under the limit.
- Scheduler Core → Bind Pod: binds · Performs the write-side pod binding after choosing a node.
- Bind Pod → K8s API: writes · Sends the bind request through the Kubernetes API.
- Pod 1 → K8s API: creates pending pod · Example workload enters the cluster for scheduling.
- Pod 2 → K8s API: creates pending pod · Example workload enters the cluster for scheduling.
- Pod 3 → K8s API: creates pending pod · Example workload enters the cluster for scheduling.
- K8s API → Scheduler Pod: notifies · Scheduler observes pod state changes via the API.