# rh-aiservices-bu/spark-on-openshift architecture

> Architecture diagram of the GitHub repository rh-aiservices-bu/spark-on-openshift, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-11-09). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/rh-aiservices-bu/spark-on-openshift
- Repository: https://github.com/rh-aiservices-bu/spark-on-openshift

## Overview

This repository is a set of guides, example workloads, image recipes, and OpenShift manifests for running Spark—not an application with its own runtime code. The principal workflow is to choose operator-based or direct submission, run a Spark workload on OpenShift, and optionally retain its event logs in shared object storage for later viewing in the History Server. Separate documented capabilities include custom Spark/PySpark images and Prometheus/Grafana monitoring. Relationships below follow the README; no source excerpts were provided.

Main entry points: [`examples`](https://github.com/rh-aiservices-bu/spark-on-openshift/tree/main/examples) (Spark workloads), [`README.adoc`](https://github.com/rh-aiservices-bu/spark-on-openshift/blob/main/README.adoc) (Usage guide), [`examples/operator`](https://github.com/rh-aiservices-bu/spark-on-openshift/tree/main/examples/operator) (Operator examples), [`examples/standalone`](https://github.com/rh-aiservices-bu/spark-on-openshift/tree/main/examples/standalone) (Standalone examples)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_guide["Guidance and examples"]
  node_guide["Usage guide<br/>[README.adoc]"]
  node_operator_examples["Operator examples"]
  node_standalone_examples["Standalone examples"]
end

subgraph group_workloads["Spark workloads"]
  node_spark_operator["Spark operator"]
  node_spark_workloads["Spark workloads"]
end

subgraph group_images["Custom images"]
  node_image_recipes["Spark image recipes"]
  node_spark_images["Spark images"]
end

subgraph group_operations["Operations and monitoring"]
  node_object_storage[("Object storage")]
  node_history_server["History Server deployment"]
  node_monitoring_manifests["Monitoring manifests"]
  node_prometheus_grafana["Prometheus and Grafana"]
end

node_user(("Spark user"))

node_user -->|"chooses workflow"| node_guide
node_guide -->|"documents operator use"| node_operator_examples
node_guide -->|"documents direct submit"| node_standalone_examples
node_user -->|"applies job configuration"| node_spark_operator
node_operator_examples -->|"configures jobs"| node_spark_operator
node_user -->|"submits Spark jobs"| node_spark_workloads
node_standalone_examples -->|"demonstrates submission"| node_spark_workloads
node_image_recipes -->|"provides recipes"| node_spark_images
node_spark_workloads -->|"writes event logs"| node_object_storage
node_history_server -->|"reads event logs"| node_object_storage
node_monitoring_manifests -->|"supports monitoring"| node_prometheus_grafana
node_prometheus_grafana -->|"monitors operator"| node_spark_operator
node_prometheus_grafana -->|"monitors applications"| node_spark_workloads

click node_guide "https://github.com/rh-aiservices-bu/spark-on-openshift/blob/main/README.adoc"
click node_operator_examples "https://github.com/rh-aiservices-bu/spark-on-openshift/tree/main/examples/operator"
click node_standalone_examples "https://github.com/rh-aiservices-bu/spark-on-openshift/tree/main/examples/standalone"
click node_spark_workloads "https://github.com/rh-aiservices-bu/spark-on-openshift/tree/main/examples"
click node_image_recipes "https://github.com/rh-aiservices-bu/spark-on-openshift/tree/main/spark-images"
click node_history_server "https://github.com/rh-aiservices-bu/spark-on-openshift/blob/main/spark-history-server/spark-hs-deployment.yaml"
click node_monitoring_manifests "https://github.com/rh-aiservices-bu/spark-on-openshift/tree/main/spark-operator"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_guide,node_operator_examples,node_standalone_examples,node_user toneBlue
class node_spark_operator,node_spark_workloads toneAmber
class node_image_recipes,node_spark_images toneMint
class node_object_storage,node_history_server,node_monitoring_manifests,node_prometheus_grafana toneRose
```

## Components

### Guidance and examples

- [Usage guide](https://github.com/rh-aiservices-bu/spark-on-openshift/blob/main/README.adoc): `README.adoc`
- [Operator examples](https://github.com/rh-aiservices-bu/spark-on-openshift/tree/main/examples/operator): `examples/operator`
- [Standalone examples](https://github.com/rh-aiservices-bu/spark-on-openshift/tree/main/examples/standalone): `examples/standalone`

### Spark workloads

- **Spark operator**
- [Spark workloads](https://github.com/rh-aiservices-bu/spark-on-openshift/tree/main/examples): `examples`

### Custom images

- [Spark image recipes](https://github.com/rh-aiservices-bu/spark-on-openshift/tree/main/spark-images): `spark-images`
- **Spark images**

### Operations and monitoring

- **Object storage**
- [History Server deployment](https://github.com/rh-aiservices-bu/spark-on-openshift/blob/main/spark-history-server/spark-hs-deployment.yaml): `spark-history-server/spark-hs-deployment.yaml`
- [Monitoring manifests](https://github.com/rh-aiservices-bu/spark-on-openshift/tree/main/spark-operator): `spark-operator`
- **Prometheus and Grafana**

### Other components

- **Spark user**

## Connections

- Spark user → Usage guide: chooses workflow (evidence: `README.adoc`)
- Usage guide → Operator examples: documents operator use (evidence: `README.adoc`)
- Usage guide → Standalone examples: documents direct submit (evidence: `README.adoc`)
- Spark user → Spark operator: applies job configuration (evidence: `README.adoc`)
- Operator examples → Spark operator: configures jobs (evidence: `README.adoc`)
- Spark user → Spark workloads: submits Spark jobs (evidence: `README.adoc`)
- Standalone examples → Spark workloads: demonstrates submission (evidence: `README.adoc`)
- Spark image recipes → Spark images: provides recipes (evidence: `README.adoc`)
- Spark workloads → Object storage: writes event logs (evidence: `README.adoc`)
- History Server deployment → Object storage: reads event logs (evidence: `README.adoc`)
- Monitoring manifests → Prometheus and Grafana: supports monitoring (evidence: `README.adoc`)
- Prometheus and Grafana → Spark operator: monitors operator (evidence: `README.adoc`)
- Prometheus and Grafana → Spark workloads: monitors applications (evidence: `README.adoc`)