# yaravind/kafka-connect-jenkins architecture

> Architecture diagram of the GitHub repository yaravind/kafka-connect-jenkins, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-01-03). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/yaravind/kafka-connect-jenkins
- Repository: https://github.com/yaravind/kafka-connect-jenkins

## Overview

##### Main Components

a) External Systems:
- Jenkins CI Server
- Apache Kafka Cluster
- Zookeeper (required for Kafka)

b) Core Connector Components:
- JenkinsSourceConnector
- JenkinsSourceTask
- Jenkins Client
- Offset Management System
- Configuration Management

**[Jenkins Server] -> [Kafka Connect Jenkins Connector] -> [Kafka Cluster]**

##### Detailed Component Breakdown

**Jenkins Server**

- REST API Layer
- (/api/json endpoints)

**Kafka Connect Jenkins Connector**

- JenkinsClient
- Domain Objects (Job, Build, BuildCollection)

- JenkinsSourceConnector
- Configuration Management

- JenkinsSourceTask
- Offset Management
- Data Transform

- Storage Layer
- ReadYourWritesOffsetStorageAdapter

**Kafka Cluster**

- Topics

##### Key Interactions to Highlight

a) Data Flow:
- Jenkins REST API calls (GET requests)
- Build event processing
- Kafka topic publishing

b) Control Flow:
- Configuration management
- Task coordination
- Offset tracking

##### Technical Details

- REST API endpoints
- Configuration properties
- Topic names
- Authentication points
- Monitoring endpoints (REST API ports)

##### Implementation Notes

- The source connector's role in the Kafka Connect framework
- The task distribution mechanism
- The build event transformation process
- The offset tracking system

##### Monitoring Points

Monitoring endpoints:
- Kafka Connect REST API (port 8083)
- Jenkins API health checks
- Task status monitoring

## Diagram (Mermaid)

```mermaid
graph TB
    subgraph External["External Systems"]
        Jenkins["Jenkins CI Server"]
        Kafka["Apache Kafka Cluster"]
        Zookeeper["Zookeeper"]
    end

    subgraph Connector["Kafka Connect Jenkins Connector"]
        SourceConnector["JenkinsSourceConnector"]:::connector
        SourceTask["JenkinsSourceTask"]:::task
        JClient["Jenkins Client"]:::client
        OffsetMgmt["Offset Management System"]:::storage
        ConfigMgmt["Configuration Management"]:::config
        TaskConfig["Task Configuration"]:::config
        Version["Version Management"]:::config
        
        subgraph Domain["Domain Objects"]
            Job["Job"]:::domain
            Build["Build"]:::domain
            BuildCollection["BuildCollection"]:::domain
        end
    end

    subgraph Storage["Storage Layer"]
        RYWAdapter["ReadYourWritesOffsetStorageAdapter"]:::storage
    end

    subgraph API["Jenkins REST API Layer"]
        REST["/api/json endpoints"]:::api
    end

    subgraph KafkaLayer["Kafka Layer"]
        Topics["Kafka Topics"]:::kafka
        ConnectAPI["Kafka Connect REST API\n(Port 8083)"]:::api
    end

    %% Relationships
    Jenkins --> REST
    REST --> JClient
    JClient --> Domain
    Domain --> SourceTask
    SourceConnector --> ConfigMgmt
    SourceConnector --> TaskConfig
    SourceTask --> OffsetMgmt
    OffsetMgmt --> RYWAdapter
    SourceTask --> Topics
    Kafka --- Zookeeper
    Topics --> Kafka
    Version --> SourceConnector

    %% Click Events
    click SourceConnector "https://github.com/yaravind/kafka-connect-jenkins/blob/master/src/main/java/org/aravind/oss/kafka/connect/jenkins/JenkinsSourceConnector.java"
    click SourceTask "https://github.com/yaravind/kafka-connect-jenkins/blob/master/src/main/java/org/aravind/oss/kafka/connect/jenkins/JenkinsSourceTask.java"
    click JClient "https://github.com/yaravind/kafka-connect-jenkins/blob/master/src/main/java/org/aravind/oss/jenkins/JenkinsClient.java"
    click OffsetMgmt "https://github.com/yaravind/kafka-connect-jenkins/blob/master/src/main/java/org/aravind/oss/kafka/connect/jenkins/ReadYourWritesOffsetStorageAdapter.java"
    click ConfigMgmt "https://github.com/yaravind/kafka-connect-jenkins/blob/master/src/main/java/org/aravind/oss/kafka/connect/jenkins/JenkinsSourceConfig.java"
    click Job "https://github.com/yaravind/kafka-connect-jenkins/blob/master/src/main/java/org/aravind/oss/jenkins/domain/Job.java"
    click Build "https://github.com/yaravind/kafka-connect-jenkins/blob/master/src/main/java/org/aravind/oss/jenkins/domain/Build.java"
    click BuildCollection "https://github.com/yaravind/kafka-connect-jenkins/blob/master/src/main/java/org/aravind/oss/jenkins/domain/BuildCollection.java"
    click TaskConfig "https://github.com/yaravind/kafka-connect-jenkins/blob/master/src/main/java/org/aravind/oss/kafka/connect/jenkins/JobTaskConfigExtractor.java"
    click Version "https://github.com/yaravind/kafka-connect-jenkins/blob/master/src/main/java/org/aravind/oss/kafka/connect/jenkins/Version.java"

    %% Styling
    classDef external fill:#2374f7,stroke:#333,stroke-width:2px
    classDef connector fill:#4CAF50,stroke:#333,stroke-width:2px
    classDef task fill:#81C784,stroke:#333,stroke-width:2px
    classDef client fill:#66BB6A,stroke:#333,stroke-width:2px
    classDef storage fill:#FFD700,stroke:#333,stroke-width:2px
    classDef config fill:#A9A9A9,stroke:#333,stroke-width:2px
    classDef domain fill:#90EE90,stroke:#333,stroke-width:2px
    classDef api fill:#FF6B6B,stroke:#333,stroke-width:2px
    classDef kafka fill:#2374f7,stroke:#333,stroke-width:2px

    class Jenkins,Kafka,Zookeeper external
    class SourceConnector,Domain connector
    class SourceTask task
    class JClient client
    class OffsetMgmt,RYWAdapter storage
    class ConfigMgmt,TaskConfig,Version config
    class Job,Build,BuildCollection domain
    class REST,ConnectAPI api
    class Topics kafka
```
