# ray-project/kuberay architecture

> Architecture diagram of the GitHub repository ray-project/kuberay, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-05-10). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/ray-project/kuberay
- Repository: https://github.com/ray-project/kuberay

## Overview

1. **Overall shape**
   KubeRay is a Kubernetes-native control plane for Ray. The repo is split into several independently deployable pieces: the **core Ray operator**, an **API server layer**, a **history/log service**, a **dashboard UI**, a **kubectl plugin**, SDKs, and the Helm/manifests used to install them.

2. **Core control plane: `ray-operator/`**
   This is the main runtime. It is a Go-based Kubernetes operator that watches the Ray custom resources **RayCluster**, **RayJob**, **RayService**, and **RayCronJob**. Its controllers reconcile these objects into Kubernetes primitives such as Pods, Services, RBAC, Ingress, and autoscaling-related resources.

3. **Operator reconciliation flow**
   The operator is the authoritative manager for Ray workloads. User-authored CRs enter Kubernetes, the operator validates them, mutates/defaults via webhooks, and then continuously reconciles desired state with actual cluster state. It also owns lifecycle actions like cluster creation/deletion, job submission, service upgrades, and status updates back into CR status fields.

4. **RayCluster subsystem**
   `RayCluster` is the lowest-level workload model. The controller and shared helpers in `controllers/ray/common/` build the head/worker pod specs, services, RBAC, and optional ingress. This subsystem is also where scheduler integrations and cluster-specific behaviors live, including autoscaling and resource-related logic.

5. **RayJob subsystem**
   `RayJob` is a higher-level workflow: create a RayCluster, wait for readiness, submit a job, then optionally tear down the cluster when the job completes. The `rayjob-submitter/` package is the in-cluster component used to submit jobs into Ray. This creates a clear boundary between cluster lifecycle management and job execution.

6. **RayService subsystem**
   `RayService` couples a RayCluster with Ray Serve deployment state and handles zero-downtime upgrades/HA behavior. The controller manages service health, rollout/upgrade orchestration, and consistency between the serving graph and the underlying Ray cluster. Supporting logic lives in `controllers/ray/utils/` and the serve-related helpers.

7. **Admission, validation, and generated client layer**
   The operator includes webhook code under `pkg/webhooks/` plus generated Kubernetes clientsets/informers/listers under `pkg/client/`. This is the usual Kubernetes operator boundary: API objects are validated/mutated at admission time, then watched and reconciled using generated typed clients.

8. **APIServer: `apiserver/`**
   This is a separate Go service that exposes a simplified API for Ray resource management. It translates API-server-facing models into Ray/Kubernetes operations through internal managers, client adapters, and HTTP helpers. It is designed as an abstraction layer used by UIs and higher-level tooling rather than as the core control plane.

9. **APIServer data model and transport**
   The `proto/` directory contains the gRPC/REST interface definitions and generated Go clients for the API server. The `apiserversdk/` and `clients/python-*` packages are client-facing layers on top of those APIs, providing either higher-level builders or direct parameter-based helpers for creating clusters/jobs/templates.

10. **Dashboard: `dashboard/`**
    The dashboard is a Next.js/TypeScript frontend. It does not talk directly to Ray workloads; instead it calls backend routes that proxy to the API server and history server. Its main architectural role is presentation and user workflow orchestration for listing, creating, and managing clusters/jobs/history data.

11. **History subsystem: `historyserver/`**
    The history server is a Go service that aggregates Ray event/task/log history. It is split into a **collector** that ingests events/logs from Ray workloads and a **historyserver** that serves query APIs and debug state. Storage is abstracted behind pluggable backends such as S3, GCS, Azure Blob, Aliyun OSS, and local test implementations.

12. **Event and log flow**
    For history data, the flow is: Ray workload emits logs/events → collector reads them → collector writes to object storage → history server reads and serves them to clients/UI. This is the main async data pipeline in the repo and is distinct from the synchronous operator reconciliation path.

13. **kubectl plugin: `kubectl-plugin/`**
    The `kubectl ray` plugin is a CLI wrapper around common cluster/job operations. It talks to Kubernetes and Ray-related APIs to create, inspect, scale, submit jobs, fetch logs, and manage sessions. Architecturally, it is a user-entry tool that sits outside the control plane and drives the same CRDs/services.

14. **Packaging and deployment**
    The repo includes Helm charts for the operator, API server, and sample Ray cluster deployments, plus Kustomize manifests under each component. This means the deployment architecture is multi-component: operator, optional API server/dashboard/historyserver, and the actual Ray workloads in user namespaces.

15. **Auxiliary/experimental pieces**
    `experimental/` contains proxy-related services for HTTP/gRPC routing and authentication, while `podpool/` appears to be a separate utility/controller for pod management. These are secondary boundaries compared with the main operator/API/history/dashboard stack, but they indicate the repo also explores networking and scheduling adjacencies around Ray.

16. **How to draw the main diagram**
    The cleanest architecture is: **users/CLI/UI → API server/dashboard/operator**; **operator → Kubernetes API → Ray pods/services/CRDs**; **Ray workloads → history collector → object storage → history server/UI**. Around that, show the shared generated clients/proto, Helm/manifests as deployment assets, and the Python/CLI SDKs as external entry points.

Main entry points: [`apiserver`](https://github.com/ray-project/kuberay/tree/master/apiserver) (API Server), [`ray-operator`](https://github.com/ray-project/kuberay/tree/master/ray-operator) (Ray Operator), [`historyserver/pkg/historyserver`](https://github.com/ray-project/kuberay/tree/master/historyserver/pkg/historyserver) (History Server), [`ray-operator/controllers/ray/raycluster_controller.go`](https://github.com/ray-project/kuberay/blob/master/ray-operator/controllers/ray/raycluster_controller.go) (Cluster Ctrl)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_control_plane["Control Plane"]
  node_ray_operator["Ray Operator<br/>go operator"]
  node_raycluster_ctrl["Cluster Ctrl<br/>reconcile"]
  node_rayjob_ctrl["Job Ctrl<br/>workflow reconcile"]
  node_rayservice_ctrl["Service Ctrl<br/>serve reconcile"]
  node_webhooks["Webhooks<br/>admission"]
  node_generated_clients["Typed Clients<br/>generated kubernetes"]
  node_common_helpers["Common Helpers<br/>controller shared"]
  node_scheduler_integration["Scheduler Int.<br/>batch scheduler"]
end

subgraph group_user_facing["User Entry"]
  node_api_server["API Server<br/>go service"]
  node_api_models["Proto APIs<br/>grpc/rest schema"]
  node_api_sdk["API SDK<br/>go sdk"]
  node_python_clients["Python Clients<br/>python sdk"]
  node_python_api_client["API Client<br/>python sdk"]
  node_dashboard["Dashboard<br/>nextjs ui"]
  node_kubectl_plugin["kubectl ray<br/>cli plugin"]
end

subgraph group_history_data["History Pipeline"]
  node_history_server["History Server<br/>go service"]
  node_collector["Collector<br/>ingest pipeline"]
  node_storage_backends["Storage Backends<br/>object storage"]
end

subgraph group_delivery["Packaging"]
  node_helm_manifests["Helm Charts<br/>deployment assets"]
  node_config_assets["Config Assets<br/>ops assets"]
  node_experimental["Experimental<br/>proxy utilities"]
  node_podpool["PodPool<br/>pod controller"]
end

node_ray_operator -->|"hosts"| node_raycluster_ctrl
node_ray_operator -->|"hosts"| node_rayjob_ctrl
node_ray_operator -->|"hosts"| node_rayservice_ctrl
node_ray_operator -->|"admits"| node_webhooks
node_ray_operator -->|"watches"| node_generated_clients
node_raycluster_ctrl -->|"uses"| node_common_helpers
node_rayjob_ctrl -->|"uses"| node_common_helpers
node_rayservice_ctrl -->|"uses"| node_common_helpers
node_raycluster_ctrl -->|"delegates"| node_scheduler_integration
node_rayjob_ctrl -->|"submits via"| node_api_server
node_rayservice_ctrl -->|"coordinates with"| node_api_server
node_dashboard -->|"calls"| node_api_server
node_dashboard -->|"queries"| node_history_server
node_kubectl_plugin -->|"drives"| node_ray_operator
node_kubectl_plugin -->|"calls"| node_api_server
node_api_server -->|"implements"| node_api_models
node_api_sdk -->|"targets"| node_api_server
node_python_api_client -->|"targets"| node_api_server
node_python_clients -->|"targets"| node_api_server
node_history_server -->|"reads"| node_collector
node_collector -->|"writes"| node_storage_backends
node_history_server -->|"reads"| node_storage_backends
node_collector -->|"observes"| node_ray_operator
node_helm_manifests -->|"installs"| node_ray_operator
node_helm_manifests -->|"installs"| node_api_server
node_config_assets -->|"supports"| node_dashboard
node_config_assets -->|"supports"| node_history_server
node_experimental -.->|"extends"| node_api_server
node_podpool -.->|"adjacent"| node_ray_operator

click node_ray_operator "https://github.com/ray-project/kuberay/tree/master/ray-operator"
click node_raycluster_ctrl "https://github.com/ray-project/kuberay/blob/master/ray-operator/controllers/ray/raycluster_controller.go"
click node_rayjob_ctrl "https://github.com/ray-project/kuberay/blob/master/ray-operator/controllers/ray/rayjob_controller.go"
click node_rayservice_ctrl "https://github.com/ray-project/kuberay/blob/master/ray-operator/controllers/ray/rayservice_controller.go"
click node_webhooks "https://github.com/ray-project/kuberay/tree/master/ray-operator/pkg/webhooks/v1"
click node_generated_clients "https://github.com/ray-project/kuberay/tree/master/ray-operator/pkg/client"
click node_common_helpers "https://github.com/ray-project/kuberay/tree/master/ray-operator/controllers/ray/common"
click node_scheduler_integration "https://github.com/ray-project/kuberay/tree/master/ray-operator/controllers/ray/batchscheduler"
click node_api_server "https://github.com/ray-project/kuberay/tree/master/apiserver"
click node_api_models "https://github.com/ray-project/kuberay/tree/master/proto"
click node_api_sdk "https://github.com/ray-project/kuberay/tree/master/apiserversdk"
click node_python_clients "https://github.com/ray-project/kuberay/tree/master/clients/python-client"
click node_python_api_client "https://github.com/ray-project/kuberay/tree/master/clients/python-apiserver-client"
click node_dashboard "https://github.com/ray-project/kuberay/tree/master/dashboard"
click node_kubectl_plugin "https://github.com/ray-project/kuberay/tree/master/kubectl-plugin"
click node_history_server "https://github.com/ray-project/kuberay/tree/master/historyserver/pkg/historyserver"
click node_collector "https://github.com/ray-project/kuberay/tree/master/historyserver/pkg/collector"
click node_storage_backends "https://github.com/ray-project/kuberay/tree/master/historyserver/pkg/storage"
click node_helm_manifests "https://github.com/ray-project/kuberay/tree/master/helm-chart"
click node_config_assets "https://github.com/ray-project/kuberay/tree/master/config"
click node_experimental "https://github.com/ray-project/kuberay/tree/master/experimental"
click node_podpool "https://github.com/ray-project/kuberay/tree/master/podpool"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_ray_operator,node_raycluster_ctrl,node_rayjob_ctrl,node_rayservice_ctrl,node_webhooks,node_generated_clients,node_common_helpers,node_scheduler_integration toneBlue
class node_api_server,node_api_models,node_api_sdk,node_python_clients,node_python_api_client,node_dashboard,node_kubectl_plugin toneAmber
class node_history_server,node_collector,node_storage_backends toneMint
class node_helm_manifests,node_config_assets,node_experimental,node_podpool toneRose
```

## Components

### Control Plane

Core Kubernetes-native reconciliation and API services

- [Ray Operator](https://github.com/ray-project/kuberay/tree/master/ray-operator): go operator · Main Kubernetes controller for Ray CRDs and workload reconciliation · `ray-operator`
- [Cluster Ctrl](https://github.com/ray-project/kuberay/blob/master/ray-operator/controllers/ray/raycluster_controller.go): reconcile · Reconciles RayCluster into pods, services, RBAC, ingress, and autoscaling behavior · `ray-operator/controllers/ray/raycluster_controller.go`
- [Job Ctrl](https://github.com/ray-project/kuberay/blob/master/ray-operator/controllers/ray/rayjob_controller.go): workflow reconcile · Creates clusters, submits jobs, and handles job lifecycle/teardown · `ray-operator/controllers/ray/rayjob_controller.go`
- [Service Ctrl](https://github.com/ray-project/kuberay/blob/master/ray-operator/controllers/ray/rayservice_controller.go): serve reconcile · Manages RayService rollouts, health, and zero-downtime upgrades · `ray-operator/controllers/ray/rayservice_controller.go`
- [Webhooks](https://github.com/ray-project/kuberay/tree/master/ray-operator/pkg/webhooks/v1): admission · Validation and mutation boundary for Ray API objects · `ray-operator/pkg/webhooks/v1`
- [Typed Clients](https://github.com/ray-project/kuberay/tree/master/ray-operator/pkg/client): generated kubernetes · Generated clientsets, informers, and listers for Ray CRDs · `ray-operator/pkg/client`
- [Common Helpers](https://github.com/ray-project/kuberay/tree/master/ray-operator/controllers/ray/common): controller shared · Shared pod, service, RBAC, ingress, and job assembly logic · `ray-operator/controllers/ray/common`
- [Scheduler Int.](https://github.com/ray-project/kuberay/tree/master/ray-operator/controllers/ray/batchscheduler): batch scheduler · Integrations for Volcano, YuniKorn, Kai, and scheduler plugins · `ray-operator/controllers/ray/batchscheduler`

### User Entry

CLI, UI, and SDK layers used by operators and developers

- [API Server](https://github.com/ray-project/kuberay/tree/master/apiserver): go service · Separate service that exposes simplified Ray management APIs · `apiserver`
- [Proto APIs](https://github.com/ray-project/kuberay/tree/master/proto): grpc/rest schema · Shared API definitions and generated Go gateway/client code · `proto`
- [API SDK](https://github.com/ray-project/kuberay/tree/master/apiserversdk): go sdk · Go-side helper SDK for driving the API server · `apiserversdk`
- [Python Clients](https://github.com/ray-project/kuberay/tree/master/clients/python-client): python sdk · Python client libraries for clusters, jobs, and templates · `clients/python-client`
- [API Client](https://github.com/ray-project/kuberay/tree/master/clients/python-apiserver-client): python sdk · Python client for the API server model layer · `clients/python-apiserver-client`
- [Dashboard](https://github.com/ray-project/kuberay/tree/master/dashboard): nextjs ui · Web UI that lists and manages clusters, jobs, and history · `dashboard`
- [kubectl ray](https://github.com/ray-project/kuberay/tree/master/kubectl-plugin): cli plugin · Kubectl extension for cluster, job, log, and session operations · `kubectl-plugin`

### History Pipeline

Async collection and serving of Ray event and log history

- [History Server](https://github.com/ray-project/kuberay/tree/master/historyserver/pkg/historyserver): go service · Serves queried Ray event, task, and log history · `historyserver/pkg/historyserver`
- [Collector](https://github.com/ray-project/kuberay/tree/master/historyserver/pkg/collector): ingest pipeline · Collects logs and events from Ray workloads and writes history data · `historyserver/pkg/collector`
- [Storage Backends](https://github.com/ray-project/kuberay/tree/master/historyserver/pkg/storage): object storage · Pluggable persistence for history data across S3, GCS, Azure, OSS, and local · `historyserver/pkg/storage`

### Packaging

Helm, manifests, and deployment assets

- [Helm Charts](https://github.com/ray-project/kuberay/tree/master/helm-chart): deployment assets · Install packages for operator, API server, and sample Ray clusters · `helm-chart`
- [Config Assets](https://github.com/ray-project/kuberay/tree/master/config): ops assets · Grafana dashboards, Prometheus rules, samples, and install scripts · `config`
- [Experimental](https://github.com/ray-project/kuberay/tree/master/experimental): proxy utilities · Optional HTTP/gRPC proxy and auth experiments · `experimental`
- [PodPool](https://github.com/ray-project/kuberay/tree/master/podpool): pod controller · Secondary pod management utility/controller · `podpool`

## Connections

- Ray Operator → Cluster Ctrl: hosts · Operator hosts the RayCluster controller
- Ray Operator → Job Ctrl: hosts · Operator hosts the RayJob controller
- Ray Operator → Service Ctrl: hosts · Operator hosts the RayService controller
- Ray Operator → Webhooks: admits · Admission webhooks validate and mutate Ray CRs
- Ray Operator → Typed Clients: watches · Controllers rely on generated typed clients and informers
- Cluster Ctrl → Common Helpers: uses · Cluster reconciliation builds workloads through shared helpers
- Job Ctrl → Common Helpers: uses · Job reconciliation reuses cluster and job assembly helpers
- Service Ctrl → Common Helpers: uses · Service reconciliation shares resource-building logic
- Cluster Ctrl → Scheduler Int.: delegates · Cluster placement can delegate to scheduler integrations
- Job Ctrl → API Server: submits via · Jobs may be submitted through the API server layer
- Service Ctrl → API Server: coordinates with · Service rollout state is exposed through API server flows
- Dashboard → API Server: calls · UI uses backend routes that proxy to the API server
- Dashboard → History Server: queries · UI retrieves history data through proxy routes
- kubectl ray → Ray Operator: drives · Kubectl plugin operates on Ray CRDs and cluster state
- kubectl ray → API Server: calls · Plugin can use API server-backed operations
- API Server → Proto APIs: implements · Service implements the proto and REST definitions
- API SDK → API Server: targets · Go SDK is a client for API server workflows
- API Client → API Server: targets · Python API client wraps API server operations
- Python Clients → API Server: targets · Python client helpers drive API server-backed cluster and job actions
- History Server → Collector: reads · History server depends on collected event and log data
- Collector → Storage Backends: writes · Collector persists events and logs to storage backends
- History Server → Storage Backends: reads · Query service reads persisted history from storage
- Collector → Ray Operator: observes · Collector runs against Ray workloads created by the operator
- Helm Charts → Ray Operator: installs · Helm charts package the operator deployment
- Helm Charts → API Server: installs · Helm charts package the API server deployment
- Config Assets → Dashboard: supports · UI dashboards and sample configs support the dashboard workflows
- Config Assets → History Server: supports · Prometheus and Grafana assets support the history service
- Experimental → API Server: extends · Experimental proxy utilities sit alongside the API-facing services
- PodPool → Ray Operator: adjacent · Separate pod-management utility adjacent to the main control plane
