# gokcesurenkok/ha-flink-cluster-k8s architecture

> Architecture diagram of the GitHub repository gokcesurenkok/ha-flink-cluster-k8s, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-01-15). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/gokcesurenkok/ha-flink-cluster-k8s
- Repository: https://github.com/gokcesurenkok/ha-flink-cluster-k8s

## Overview

##### Project Type Analysis
This is a Kubernetes-based infrastructure project specifically focused on deploying a highly available Apache Flink cluster. The architecture should emphasize the distributed nature of Flink and its integration with Kubernetes orchestration.

##### Key Components

a. Kubernetes Layer:
- Kubernetes cluster boundary
- ConfigMaps (flink-cfg, hadoop-cfg)
- Services layer
- Deployments/StatefulSets

b. Flink Components:
- JobManager StatefulSet (for HA)
- TaskManager Deployment
- Replication/scaling

c. External Dependencies:
- ZooKeeper (for HA coordination)
- HDFS (for storage)

##### Specific Elements

a. Kubernetes Components:
**┌─ ConfigMaps ─────┐**
- flink-cfg     │
- hadoop-cfg    │
- ┘

**┌─ Services ──────┐**
- flink-svc    │
- ┘

**┌─ Deployments ───┐**
- JobManager   │
- TaskManager  │
- ┘

b. High Availability Setup:
- ZooKeeper connection for leader election
- Multiple JobManager instances
- TaskManager scaling capabilities

## Diagram (Mermaid)

```mermaid
graph TB
    subgraph Kubernetes Cluster
        subgraph "Configuration Layer"
            CM1["Flink ConfigMap"]:::configmap
            CM2["Hadoop ConfigMap"]:::configmap
        end

        subgraph "Service Layer"
            SVC["Flink Service"]:::service
        end

        subgraph "Application Layer"
            subgraph "JobManager HA"
                JM1["JobManager Pod 1"]:::jobmanager
                JM2["JobManager Pod 2"]:::jobmanager
                JM3["JobManager Pod 3"]:::jobmanager
            end

            subgraph "TaskManager Scale"
                TM1["TaskManager Pod 1"]:::taskmanager
                TM2["TaskManager Pod 2"]:::taskmanager
                TM3["TaskManager Pod n"]:::taskmanager
            end
        end

        subgraph "External Services"
            ZK["ZooKeeper"]:::external
            HDFS["HDFS Storage"]:::external
        end
    end

    %% Relationships
    CM1 --> JM1
    CM1 --> JM2
    CM1 --> JM3
    CM1 --> TM1
    CM1 --> TM2
    CM1 --> TM3

    CM2 --> HDFS

    SVC --> JM1
    SVC --> JM2
    SVC --> JM3

    JM1 -.-> ZK
    JM2 -.-> ZK
    JM3 -.-> ZK

    JM1 --> TM1
    JM1 --> TM2
    JM1 --> TM3

    %% Click Events
    click CM1 "https://github.com/gokcesurenkok/ha-flink-cluster-k8s/blob/master/configmap/flink-cfg.yaml"
    click CM2 "https://github.com/gokcesurenkok/ha-flink-cluster-k8s/blob/master/configmap/hadoop-cfg.yaml"
    click JM1 "https://github.com/gokcesurenkok/ha-flink-cluster-k8s/blob/master/deployment/job-manager-sts.yml"
    click TM1 "https://github.com/gokcesurenkok/ha-flink-cluster-k8s/blob/master/deployment/task-manager-dep.yml"
    click SVC "https://github.com/gokcesurenkok/ha-flink-cluster-k8s/blob/master/services/flink-svc.yml"

    %% Styling
    classDef configmap fill:#ffeb3b,stroke:#000
    classDef service fill:#03a9f4,stroke:#000
    classDef jobmanager fill:#4caf50,stroke:#000
    classDef taskmanager fill:#81c784,stroke:#000
    classDef external fill:#9c27b0,stroke:#000

    %% Legend
    subgraph Legend
        L1["ConfigMap"]:::configmap
        L2["Service"]:::service
        L3["JobManager"]:::jobmanager
        L4["TaskManager"]:::taskmanager
        L5["External Service"]:::external
    end
```
