# risingwavelabs/risingwave-operator architecture

> Architecture diagram of the GitHub repository risingwavelabs/risingwave-operator, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-04-24). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/risingwavelabs/risingwave-operator
- Repository: https://github.com/risingwavelabs/risingwave-operator

## Overview

##### Project Type and Purpose
- This is a Kubernetes Operator (written in Go using Kubebuilder/controller-runtime) whose job is to manage RisingWave clusters on Kubernetes.
- It defines CustomResourceDefinitions (CRDs) for RisingWave and RisingWaveScaleView, watches for these custom resources, and reconciles desired state by creating/updating native Kubernetes resources (StatefulSets, Deployments, Services, PVCs, ConfigMaps, Webhooks).

##### Key Top-Level Directories
- apis/…: CRD type definitions under `risingwave/v1alpha1`
- cmd/manager: operator entrypoint (manager.go)
- pkg/controller: reconciliation logic for RisingWave and ScaleView
- pkg/webhook: mutating & validating webhook handlers
- config/: all Kustomize manifests for CRDs, RBAC, manager, webhook, samples
- ci/, .github/: build/test/release pipelines
- test/ and pytest/: e2e and unit tests

##### Main Components
a. Kubernetes Cluster
- API Server & etcd (built-in Kubernetes control plane)
- Cert-Manager (external dependency for TLS/secrets)
b. CustomResourceDefinitions (CRDs)
- RisingWave
- RisingWaveScaleView
c. Operator Pod (“risingwave-operator-controller-manager”)
- Controller Manager (root manager.Process)
- RisingWaveController (pkg/controller/risingwave_controller.go)
- ScaleViewController (pkg/controller/risingwave_scale_view_controller.go)
- Webhook Server (pkg/webhook/*.go)
- Mutating & Validating Webhooks for RisingWave/ScaleView
- Metrics Recorder (pkg/metrics)
d. Child Resources Managed by the Operator
- StatefulSet(s): risingwave-meta (etcd), risingwave-compute
- Deployment(s): risingwave-frontend, compactor, scale-view workers
- Services: frontend, meta, metrics, webhook-service
- PersistentVolumeClaims: for etcd metadata, MinIO state store, etc.
- ConfigMaps: generated configurations, controller config
e. External Components (optional, if used in sample includes)
- MinIO or other object store (for state store)
- PVC-backed Postgres or SQLite (for metadata store)

##### Relationships & Data Flows
- CRD Registration → API Server
- User/Subsystem creates a RisingWave CR → API Server
- Operator’s Manager watches CRs (informer cache) → Reconciliation loops
- Reconciler issues CRUD calls to API Server → Creates/Updates StatefulSets, Deployments, Services, PVCs
- Webhook Server hooked into API Server admission → Validates/mutates CR requests
- Operator records metrics → Prometheus (if monitoring configured)

##### Architectural Patterns & Principles
- Operator Pattern / Controller Pattern:
- Single control loop per resource type
- Desired state in CR → Actual state in cluster → Reconcile
- Modularization: separate packages for API types, controllers, webhooks, metrics
- GitOps/Kustomize: all k8s manifests kept in config/ and docs/
- CI/CD: GitHub Actions and Buildkite pipelines build Docker image, run unit/e2e tests

##### Technologies & Frameworks
- Go + Kubebuilder/controller-runtime for controllers & webhooks
- Kustomize for Kubernetes manifest templating
- cert-manager for TLS certificate provisioning
- Prometheus metrics using controller-runtime instrumentation
- Helm (optional install path)

## Diagram (Mermaid)

```mermaid
flowchart TD
    %% Kubernetes Cluster
    subgraph "Kubernetes Cluster"
        API["API Server & etcd"]:::k8s
        CM["Cert-Manager"]:::external
    end

    %% CRDs and API Types
    subgraph "CustomResourceDefinitions"
        RWCRD["RisingWave CRD"]:::crd
        SVCRD["RisingWaveScaleView CRD"]:::crd
        TYPES["risingwave_types.go"]:::crd
    end

    %% Operator Pod
    subgraph "Operator Pod: risingwave-operator-controller-manager"
        MAN["Operator Entrypoint\ncmd/manager/manager.go"]:::internals
        RWCTR["RisingWaveController\npkg/controller/risingwave_controller.go"]:::internals
        SVCTR["ScaleViewController\npkg/controller/risingwave_scale_view_controller.go"]:::internals
        WEB["Webhook Server\npkg/webhook/webhook.go"]:::internals
        MWEB["RW Mutating Webhook\npkg/webhook/risingwave_mutating_webhook.go"]:::internals
        VWEB["RW Validating Webhook\npkg/webhook/risingwave_validating_webhook.go"]:::internals
        SMUT["ScaleView Mutating Webhook\npkg/webhook/risingwave_scale_view_mutating_webhook.go"]:::internals
        SVAL["ScaleView Validating Webhook\npkg/webhook/risingwave_scale_view_validating_webhook.go"]:::internals
        MREC_C["Metrics Recorder (Controller)\npkg/metrics/controller_metrics_recorder.go"]:::internals
        MREC_W["Metrics Recorder (Webhook)\npkg/metrics/webhook_metrics_recorder.go"]:::internals
        FACT["Factory\npkg/factory/risingwave_object_factory.go"]:::internals
        MANIMPL["Controller Manager Impl\npkg/manager/risingwave_controller_manager_impl.go"]:::internals
        SVIMPL["ScaleView Manager Impl\npkg/manager/risingwave_scale_view_controller_manager_impl.go"]:::internals
        UTILS["Utilities\npkg/utils/apps.go, pkg/utils/common.go"]:::internals
    end

    %% Child Resources
    subgraph "Child Resources"
        STSET["StatefulSets\n(etcd, compute)"]:::k8s
        DEP["Deployments\n(frontend, compactor)"]:::k8s
        SVC["Services\n(frontend, meta, metrics, webhook)"]:::k8s
        PVC["PersistentVolumeClaims"]:::k8s
        CMAP["ConfigMaps"]:::k8s
    end

    %% External Services and Monitoring
    subgraph "External Services & CI/CD"
        MINIO["MinIO\n(object store)"]:::external
        PG["Postgres/SQLite\n(metadata store)"]:::external
        PRO["Prometheus"]:::external
        GH["GitHub Actions\n(ci/, .github/)"]:::external
        TEST["E2E Tests\npytest/test_installation.py"]:::external
    end

    %% Configuration & Manifests
    subgraph "Configuration & Manifests"
        KCRD["CRD Manifests\nconfig/crd/bases/risingwave.risingwavelabs.com_risingwaves.yaml"]:::config
        KSVCRD["ScaleView CRD Manifests\nconfig/crd/bases/risingwave.risingwavelabs.com_risingwavescaleviews.yaml"]:::config
        MANCFG["Manager Config & RBAC\nconfig/manager/manager.yaml"]:::config
        WSVC["Webhook Service\nconfig/webhook/service.yaml"]:::config
        CERT["Certificates\nconfig/certmanager/certificate.yaml"]:::config
        SAMPLES["Samples\nconfig/samples/risingwave_v1alpha1_risingwave.yaml"]:::config
    end

    %% Relationships
    User["User/Subsystem"]:::external -->|"creates CR"| API
    API -->|"register CRDs"| RWCRD
    API -->|"register CRDs"| SVCRD
    RWCRD -->|"watch"| RWCTR
    SVCRD -->|"watch"| SVCTR
    RWCTR -->|"CRUD"| API
    SVCTR -->|"CRUD"| API
    API -->|"admission request"| WEB
    WEB -->|"mutation/validation"| API
    RWCTR --> STSET
    RWCTR --> DEP
    RWCTR --> SVC
    RWCTR --> PVC
    RWCTR --> CMAP
    MREC_C -->|"metrics"| PRO
    MREC_W -->|"metrics"| PRO
    GH -->|"build & test"| MAN

    %% Kustomize connections
    KCRD -.-> RWCRD
    KSVCRD -.-> SVCRD
    MANCFG -.-> MAN
    WSVC -.-> WEB
    CERT -.-> WEB
    SAMPLES -.-> RWCRD

    %% Click Events
    click RWCRD "https://github.com/risingwavelabs/risingwave-operator/tree/main/apis/risingwave/v1alpha1/"
    click SVCRD "https://github.com/risingwavelabs/risingwave-operator/tree/main/apis/risingwave/v1alpha1/"
    click TYPES "https://github.com/risingwavelabs/risingwave-operator/blob/main/apis/risingwave/v1alpha1/risingwave_types.go"
    click RWCTR "https://github.com/risingwavelabs/risingwave-operator/blob/main/pkg/controller/risingwave_controller.go"
    click SVCTR "https://github.com/risingwavelabs/risingwave-operator/blob/main/pkg/controller/risingwave_scale_view_controller.go"
    click MAN "https://github.com/risingwavelabs/risingwave-operator/blob/main/cmd/manager/manager.go"
    click WEB "https://github.com/risingwavelabs/risingwave-operator/blob/main/pkg/webhook/webhook.go"
    click MWEB "https://github.com/risingwavelabs/risingwave-operator/blob/main/pkg/webhook/risingwave_mutating_webhook.go"
    click VWEB "https://github.com/risingwavelabs/risingwave-operator/blob/main/pkg/webhook/risingwave_validating_webhook.go"
    click SMUT "https://github.com/risingwavelabs/risingwave-operator/blob/main/pkg/webhook/risingwave_scale_view_mutating_webhook.go"
    click SVAL "https://github.com/risingwavelabs/risingwave-operator/blob/main/pkg/webhook/risingwave_scale_view_validating_webhook.go"
    click MREC_C "https://github.com/risingwavelabs/risingwave-operator/blob/main/pkg/metrics/controller_metrics_recorder.go"
    click MREC_W "https://github.com/risingwavelabs/risingwave-operator/blob/main/pkg/metrics/webhook_metrics_recorder.go"
    click FACT "https://github.com/risingwavelabs/risingwave-operator/blob/main/pkg/factory/risingwave_object_factory.go"
    click MANIMPL "https://github.com/risingwavelabs/risingwave-operator/blob/main/pkg/manager/risingwave_controller_manager_impl.go"
    click SVIMPL "https://github.com/risingwavelabs/risingwave-operator/blob/main/pkg/manager/risingwave_scale_view_controller_manager_impl.go"
    click UTILS "https://github.com/risingwavelabs/risingwave-operator/blob/main/pkg/utils/apps.go"
    click UTILS "https://github.com/risingwavelabs/risingwave-operator/blob/main/pkg/utils/common.go"
    click TEST "https://github.com/risingwavelabs/risingwave-operator/blob/main/pytest/test_installation.py"
    click KCRD "https://github.com/risingwavelabs/risingwave-operator/blob/main/config/crd/bases/risingwave.risingwavelabs.com_risingwaves.yaml"
    click KSVCRD "https://github.com/risingwavelabs/risingwave-operator/blob/main/config/crd/bases/risingwave.risingwavelabs.com_risingwavescaleviews.yaml"
    click MANCFG "https://github.com/risingwavelabs/risingwave-operator/blob/main/config/manager/manager.yaml"
    click WSVC "https://github.com/risingwavelabs/risingwave-operator/blob/main/config/webhook/service.yaml"
    click CERT "https://github.com/risingwavelabs/risingwave-operator/blob/main/config/certmanager/certificate.yaml"
    click SAMPLES "https://github.com/risingwavelabs/risingwave-operator/blob/main/config/samples/risingwave_v1alpha1_risingwave.yaml"

    %% Styles
    classDef k8s fill:#cce5ff,stroke:#0066cc
    classDef internals fill:#d5f5e3,stroke:#27ae60
    classDef crd fill:#fcf8e3,stroke:#f1c40f
    classDef external fill:#fdebd0,stroke:#e67e22
    classDef config fill:#ebdef0,stroke:#8e44ad
```
