How yaravind/kafka-connect-jenkins works
Main Components
a) External Systems:
• Jenkins CI Server
• Apache Kafka Cluster
• Zookeeper (required for Kafka)
b) Core Connector Components:
• JenkinsSourceConnector
• JenkinsSourceTask
• Jenkins Client
• Offset Management System
• Configuration Management
[Jenkins Server] -> [Kafka Connect Jenkins Connector] -> [Kafka Cluster]
Detailed Component Breakdown
Jenkins Server
• REST API Layer
• (/api/json endpoints)
Kafka Connect Jenkins Connector
• JenkinsClient
• Domain Objects (Job, Build, BuildCollection)
• JenkinsSourceConnector
• Configuration Management
• JenkinsSourceTask
• Offset Management
• Data Transform
• Storage Layer
• ReadYourWritesOffsetStorageAdapter
Kafka Cluster
• Topics
Key Interactions to Highlight
a) Data Flow:
• Jenkins REST API calls (GET requests)
• Build event processing
• Kafka topic publishing
b) Control Flow:
• Configuration management
• Task coordination
• Offset tracking
Technical Details
• REST API endpoints
• Configuration properties
• Topic names
• Authentication points
• Monitoring endpoints (REST API ports)
Implementation Notes
• The source connector's role in the Kafka Connect framework
• The task distribution mechanism
• The build event transformation process
• The offset tracking system
Monitoring Points
Monitoring endpoints:
• Kafka Connect REST API (port 8083)
• Jenkins API health checks
• Task status monitoring