# infobip/kafkistry architecture

> Architecture diagram of the GitHub repository infobip/kafkistry, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2024-12-30). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/infobip/kafkistry
- Repository: https://github.com/infobip/kafkistry

## Overview

##### Main Components Overview

The system should be represented with these primary components:

- Kafkistry                                           |

- Web UI/API   |  |  Core Service  |  | State Scraping|  |  Repository   |          |
- Layer        |  |    Logic       |  |    Service    |  |   Layer       |          |

- Security    |  |    Auditing    |  |   Autopilot   |  |     SQL       |         |
- Module      |  |    Module      |  |    Module     |  |    Module     |         |

- ↕                                  ↕                    ↕

- Web Clients |                    | Kafka Clusters|      |     Git    |

##### Component Details

a. Web UI/API Layer (kafkistry-web, kafkistry-web-api):
- REST endpoints for all operations
- Thymeleaf templates for web interface
- Security and authentication handlers
- Static resources (JS, CSS)

b. Core Service Logic (kafkistry-service-logic):
- Topic management
- ACL management
- Consumer groups management
- Quota management
- Cluster management
- Resource analysis

c. State Scraping Service (kafkistry-state-scraping):
- Kafka cluster state monitoring
- Metadata collection
- Status comparison
- Metrics gathering

d. Repository Layer (kafkistry-repository):
- Git integration
- File system storage
- Configuration versioning
- Change history tracking

##### Supporting Modules

e. Security Module (kafkistry-security):
- Authentication
- Authorization
- User management
- Access control

f. Auditing Module (kafkistry-auditing):
- Operation logging
- Change tracking
- Slack notifications
- Audit trail

g. Autopilot Module (kafkistry-autopilot):
- Automated operations
- Action scheduling
- State monitoring
- Self-healing capabilities

h. SQL Module (kafkistry-sql):
- Metadata querying
- Analytics
- Reporting capabilities

##### External Interactions

- Kafka Clusters |     |    Git Repository |     |    Web Browsers  |

- ↑                        ↑                         ↑

- ↓                        ↓                         ↓

- Kafka Connector |     |     Repository    |     |    Web Server   |
- Module       |     |      Module       |     |     Module      |

##### Additional Elements

- Monitoring/metrics flow
- Security boundaries
- High-availability components
- Critical paths
- Data persistence points

##### Implementation Details

- Written in Kotlin
- Spring Boot based
- Uses Maven for build management
- Supports multiple Kafka clusters
- Git-based configuration storage
- Web-based UI with REST API

This system design accurately represents Kafkistry's architecture as a centralized Kafka management and monitoring solution, highlighting its key features of multi-cluster management, security, auditing, and automation capabilities.

## Diagram (Mermaid)

```mermaid
flowchart TB
    %% Styles
    classDef external fill:#ddd,stroke:#333
    classDef core fill:#f9f,stroke:#333
    classDef module fill:#bbf,stroke:#333
    classDef layer fill:#bfb,stroke:#333
    
    %% External Systems
    extKafka[("Kafka Clusters")]:::external
    extGit[("Git Repository")]:::external
    extWeb["Web Clients"]:::external
    
    %% Main Application Layers
    subgraph Kafkistry
        %% Web Layer
        subgraph WebLayer["Web Layer"]
            webUI["Web UI"]
            webAPI["Web API"]
        end
        click webUI "https://github.com/infobip/kafkistry/tree/master/kafkistry-web/"
        click webAPI "https://github.com/infobip/kafkistry/tree/master/kafkistry-web-api/"
        
        %% Core Components
        subgraph CoreComponents["Core Components"]
            serviceLogic["Service Logic"]
            stateScraping["State Scraping Service"]
            repoLayer["Repository Layer"]
            dataModels["Data Models"]
            common["Common Utilities"]
        end
        click serviceLogic "https://github.com/infobip/kafkistry/tree/master/kafkistry-service-logic/"
        click stateScraping "https://github.com/infobip/kafkistry/tree/master/kafkistry-state-scraping/"
        click repoLayer "https://github.com/infobip/kafkistry/tree/master/kafkistry-repository/"
        click dataModels "https://github.com/infobip/kafkistry/tree/master/kafkistry-model/"
        click common "https://github.com/infobip/kafkistry/tree/master/kafkistry-common/"
        
        %% Supporting Modules
        subgraph SupportModules["Supporting Modules"]
            security["Security Module"]
            auditing["Auditing Module"]
            autopilot["Autopilot Module"]
            sql["SQL Module"]
            kafkaConnector["Kafka Connector"]
        end
        click security "https://github.com/infobip/kafkistry/tree/master/kafkistry-security/"
        click auditing "https://github.com/infobip/kafkistry/tree/master/kafkistry-auditing/"
        click autopilot "https://github.com/infobip/kafkistry/tree/master/kafkistry-autopilot/"
        click sql "https://github.com/infobip/kafkistry/tree/master/kafkistry-sql/"
        click kafkaConnector "https://github.com/infobip/kafkistry/tree/master/kafkistry-kafka-connector/"
    end
    
    %% Relationships
    extWeb <--> webUI
    extWeb <--> webAPI
    webUI --> serviceLogic
    webAPI --> serviceLogic
    serviceLogic --> dataModels
    serviceLogic --> common
    stateScraping --> kafkaConnector
    kafkaConnector <--> extKafka
    repoLayer <--> extGit
    serviceLogic --> repoLayer
    serviceLogic --> stateScraping
    security --> serviceLogic
    auditing --> serviceLogic
    autopilot --> serviceLogic
    sql --> serviceLogic
    
    %% Legend
    subgraph Legend
        extSystem[("External System")]:::external
        coreComp["Core Component"]:::core
        supportMod["Supporting Module"]:::module
        appLayer["Application Layer"]:::layer
    end
    
    %% Styles for components
    class extKafka,extGit,extWeb external
    class serviceLogic,stateScraping,repoLayer,dataModels,common core
    class security,auditing,autopilot,sql,kafkaConnector module
    class webUI,webAPI layer
```
