How ray-project/kuberay works
1. Overall shape
KubeRay is a Kubernetes-native control plane for Ray. The repo is split into several independently deployable pieces: the core Ray operator, an API server layer, a history/log service, a dashboard UI, a kubectl plugin, SDKs, and the Helm/manifests used to install them.
2. Core control plane: ray-operator/
This is the main runtime. It is a Go-based Kubernetes operator that watches the Ray custom resources RayCluster, RayJob, RayService, and RayCronJob. Its controllers reconcile these objects into Kubernetes primitives such as Pods, Services, RBAC, Ingress, and autoscaling-related resources.
3. Operator reconciliation flow
The operator is the authoritative manager for Ray workloads. User-authored CRs enter Kubernetes, the operator validates them, mutates/defaults via webhooks, and then continuously reconciles desired state with actual cluster state. It also owns lifecycle actions like cluster creation/deletion, job submission, service upgrades, and status updates back into CR status fields.
4. RayCluster subsystem
RayCluster is the lowest-level workload model. The controller and shared helpers in controllers/ray/common/ build the head/worker pod specs, services, RBAC, and optional ingress. This subsystem is also where scheduler integrations and cluster-specific behaviors live, including autoscaling and resource-related logic.
5. RayJob subsystem
RayJob is a higher-level workflow: create a RayCluster, wait for readiness, submit a job, then optionally tear down the cluster when the job completes. The rayjob-submitter/ package is the in-cluster component used to submit jobs into Ray. This creates a clear boundary between cluster lifecycle management and job execution.
6. RayService subsystem
RayService couples a RayCluster with Ray Serve deployment state and handles zero-downtime upgrades/HA behavior. The controller manages service health, rollout/upgrade orchestration, and consistency between the serving graph and the underlying Ray cluster. Supporting logic lives in controllers/ray/utils/ and the serve-related helpers.
7. Admission, validation, and generated client layer
The operator includes webhook code under pkg/webhooks/ plus generated Kubernetes clientsets/informers/listers under pkg/client/. This is the usual Kubernetes operator boundary: API objects are validated/mutated at admission time, then watched and reconciled using generated typed clients.
8. APIServer: apiserver/
This is a separate Go service that exposes a simplified API for Ray resource management. It translates API-server-facing models into Ray/Kubernetes operations through internal managers, client adapters, and HTTP helpers. It is designed as an abstraction layer used by UIs and higher-level tooling rather than as the core control plane.
9. APIServer data model and transport
The proto/ directory contains the gRPC/REST interface definitions and generated Go clients for the API server. The apiserversdk/ and clients/python-* packages are client-facing layers on top of those APIs, providing either higher-level builders or direct parameter-based helpers for creating clusters/jobs/templates.
10. Dashboard: dashboard/
The dashboard is a Next.js/TypeScript frontend. It does not talk directly to Ray workloads; instead it calls backend routes that proxy to the API server and history server. Its main architectural role is presentation and user workflow orchestration for listing, creating, and managing clusters/jobs/history data.
11. History subsystem: historyserver/
The history server is a Go service that aggregates Ray event/task/log history. It is split into a collector that ingests events/logs from Ray workloads and a historyserver that serves query APIs and debug state. Storage is abstracted behind pluggable backends such as S3, GCS, Azure Blob, Aliyun OSS, and local test implementations.
12. Event and log flow
For history data, the flow is: Ray workload emits logs/events → collector reads them → collector writes to object storage → history server reads and serves them to clients/UI. This is the main async data pipeline in the repo and is distinct from the synchronous operator reconciliation path.
13. kubectl plugin: kubectl-plugin/
The kubectl ray plugin is a CLI wrapper around common cluster/job operations. It talks to Kubernetes and Ray-related APIs to create, inspect, scale, submit jobs, fetch logs, and manage sessions. Architecturally, it is a user-entry tool that sits outside the control plane and drives the same CRDs/services.
14. Packaging and deployment
The repo includes Helm charts for the operator, API server, and sample Ray cluster deployments, plus Kustomize manifests under each component. This means the deployment architecture is multi-component: operator, optional API server/dashboard/historyserver, and the actual Ray workloads in user namespaces.
15. Auxiliary/experimental pieces
experimental/ contains proxy-related services for HTTP/gRPC routing and authentication, while podpool/ appears to be a separate utility/controller for pod management. These are secondary boundaries compared with the main operator/API/history/dashboard stack, but they indicate the repo also explores networking and scheduling adjacencies around Ray.
16. How to draw the main diagram
The cleanest architecture is: users/CLI/UI → API server/dashboard/operator; operator → Kubernetes API → Ray pods/services/CRDs; Ray workloads → history collector → object storage → history server/UI. Around that, show the shared generated clients/proto, Helm/manifests as deployment assets, and the Python/CLI SDKs as external entry points.
Main entry points: apiserver API Server · ray-operator Ray Operator · historyserver History Server · raycluster_controller.go Cluster Ctrl
Components 22
Control Plane
Core Kubernetes-native reconciliation and API services
- Ray Operator · Main Kubernetes controller for Ray CRDs and workload reconciliation
ray-operator - Cluster Ctrl · Reconciles RayCluster into pods, services, RBAC, ingress, and autoscaling behavior
ray-operator/controllers/ray/raycluster_controller.go - Job Ctrl · Creates clusters, submits jobs, and handles job lifecycle/teardown
ray-operator/controllers/ray/rayjob_controller.go - Service Ctrl · Manages RayService rollouts, health, and zero-downtime upgrades
ray-operator/controllers/ray/rayservice_controller.go - Webhooks · Validation and mutation boundary for Ray API objects
ray-operator/pkg/webhooks/v1 - Typed Clients · Generated clientsets, informers, and listers for Ray CRDs
ray-operator/pkg/client - Common Helpers · Shared pod, service, RBAC, ingress, and job assembly logic
ray-operator/controllers/ray/common - Scheduler Int. · Integrations for Volcano, YuniKorn, Kai, and scheduler plugins
ray-operator/controllers/ray/batchscheduler
User Entry
CLI, UI, and SDK layers used by operators and developers
- API Server · Separate service that exposes simplified Ray management APIs
apiserver - Proto APIs · Shared API definitions and generated Go gateway/client code
proto - API SDK · Go-side helper SDK for driving the API server
apiserversdk - Python Clients · Python client libraries for clusters, jobs, and templates
clients/python-client - API Client · Python client for the API server model layer
clients/python-apiserver-client - Dashboard · Web UI that lists and manages clusters, jobs, and history
dashboard - kubectl ray · Kubectl extension for cluster, job, log, and session operations
kubectl-plugin
History Pipeline
Async collection and serving of Ray event and log history
- History Server · Serves queried Ray event, task, and log history
historyserver/pkg/historyserver - Collector · Collects logs and events from Ray workloads and writes history data
historyserver/pkg/collector - Storage Backends · Pluggable persistence for history data across S3, GCS, Azure, OSS, and local
historyserver/pkg/storage
Packaging
Helm, manifests, and deployment assets
- Helm Charts · Install packages for operator, API server, and sample Ray clusters
helm-chart - Config Assets · Grafana dashboards, Prometheus rules, samples, and install scripts
config - Experimental · Optional HTTP/gRPC proxy and auth experiments
experimental - PodPool · Secondary pod management utility/controller
podpool
Connections 29
- Ray Operator → Cluster Ctrl: hosts · Operator hosts the RayCluster controller
- Ray Operator → Job Ctrl: hosts · Operator hosts the RayJob controller
- Ray Operator → Service Ctrl: hosts · Operator hosts the RayService controller
- Ray Operator → Webhooks: admits · Admission webhooks validate and mutate Ray CRs
- Ray Operator → Typed Clients: watches · Controllers rely on generated typed clients and informers
- Cluster Ctrl → Common Helpers: uses · Cluster reconciliation builds workloads through shared helpers
- Job Ctrl → Common Helpers: uses · Job reconciliation reuses cluster and job assembly helpers
- Service Ctrl → Common Helpers: uses · Service reconciliation shares resource-building logic
- Cluster Ctrl → Scheduler Int.: delegates · Cluster placement can delegate to scheduler integrations
- Job Ctrl → API Server: submits via · Jobs may be submitted through the API server layer
- Service Ctrl → API Server: coordinates with · Service rollout state is exposed through API server flows
- Dashboard → API Server: calls · UI uses backend routes that proxy to the API server
- Dashboard → History Server: queries · UI retrieves history data through proxy routes
- kubectl ray → Ray Operator: drives · Kubectl plugin operates on Ray CRDs and cluster state
- kubectl ray → API Server: calls · Plugin can use API server-backed operations
- API Server → Proto APIs: implements · Service implements the proto and REST definitions
- API SDK → API Server: targets · Go SDK is a client for API server workflows
- API Client → API Server: targets · Python API client wraps API server operations
- Python Clients → API Server: targets · Python client helpers drive API server-backed cluster and job actions
- History Server → Collector: reads · History server depends on collected event and log data
- Collector → Storage Backends: writes · Collector persists events and logs to storage backends
- History Server → Storage Backends: reads · Query service reads persisted history from storage
- Collector → Ray Operator: observes · Collector runs against Ray workloads created by the operator
- Helm Charts → Ray Operator: installs · Helm charts package the operator deployment
- Helm Charts → API Server: installs · Helm charts package the API server deployment
- Config Assets → Dashboard: supports · UI dashboards and sample configs support the dashboard workflows
- Config Assets → History Server: supports · Prometheus and Grafana assets support the history service
- Experimental → API Server: extends · Experimental proxy utilities sit alongside the API-facing services
- PodPool → Ray Operator: adjacent · Separate pod-management utility adjacent to the main control plane