How infobip/kafkistry works
Main Components Overview
The system should be represented with these primary components:
• Kafkistry |
• Web UI/API | | Core Service | | State Scraping| | Repository | |
• Layer | | Logic | | Service | | Layer | |
• Security | | Auditing | | Autopilot | | SQL | |
• Module | | Module | | Module | | Module | |
• ↕ ↕ ↕
• Web Clients | | Kafka Clusters| | Git |
Component Details
a. Web UI/API Layer (kafkistry-web, kafkistry-web-api):
• REST endpoints for all operations
• Thymeleaf templates for web interface
• Security and authentication handlers
• Static resources (JS, CSS)
b. Core Service Logic (kafkistry-service-logic):
• Topic management
• ACL management
• Consumer groups management
• Quota management
• Cluster management
• Resource analysis
c. State Scraping Service (kafkistry-state-scraping):
• Kafka cluster state monitoring
• Metadata collection
• Status comparison
• Metrics gathering
d. Repository Layer (kafkistry-repository):
• Git integration
• File system storage
• Configuration versioning
• Change history tracking
Supporting Modules
e. Security Module (kafkistry-security):
• Authentication
• Authorization
• User management
• Access control
f. Auditing Module (kafkistry-auditing):
• Operation logging
• Change tracking
• Slack notifications
• Audit trail
g. Autopilot Module (kafkistry-autopilot):
• Automated operations
• Action scheduling
• State monitoring
• Self-healing capabilities
h. SQL Module (kafkistry-sql):
• Metadata querying
• Analytics
• Reporting capabilities
External Interactions
• Kafka Clusters | | Git Repository | | Web Browsers |
• ↑ ↑ ↑
• ↓ ↓ ↓
• Kafka Connector | | Repository | | Web Server |
• Module | | Module | | Module |
Additional Elements
• Monitoring/metrics flow
• Security boundaries
• High-availability components
• Critical paths
• Data persistence points
Implementation Details
• Written in Kotlin
• Spring Boot based
• Uses Maven for build management
• Supports multiple Kafka clusters
• Git-based configuration storage
• Web-based UI with REST API
This system design accurately represents Kafkistry's architecture as a centralized Kafka management and monitoring solution, highlighting its key features of multi-cluster management, security, auditing, and automation capabilities.