# choudharylakshyaveer/wikievent-stream-kafka architecture

> Architecture diagram of the GitHub repository choudharylakshyaveer/wikievent-stream-kafka, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-04-28). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/choudharylakshyaveer/wikievent-stream-kafka
- Repository: https://github.com/choudharylakshyaveer/wikievent-stream-kafka

## Overview

##### Project type and purpose
- This is an event-driven, microservice-style Kafka streaming project built with Spring Boot.
- It consists of two independent services: a producer that reads Wikimedia live events and pushes them to a Kafka topic, and a consumer that subscribes to that topic and persists events into a MySQL database.

##### Core components and their code residence
- Parent (root) Maven module (pom.xml, docker-compose.yml)
- wikievent-producer (Spring Boot app with KafkaTemplate)
- wikievent-consumer (Spring Boot app with KafkaListener and Spring Data JPA)
- External systems: Wikimedia SSE feed, Apache Kafka (broker + Zookeeper via docker-compose), MySQL

a. Main components:
- Wikimedia Event Stream (third-party SSE)
- Producer Service (wikievent-producer)
- Kafka Cluster (broker + Zookeeper)
- Consumer Service (wikievent-consumer)
- MySQL Database
1. Producer polls or listens to Wikimedia SSE → parses events
2. Producer sends messages → Kafka topic (e.g. “wikievents”)
3. Consumer subscribes to topic → receives message batches
4. Consumer writes records → MySQL via JPA repository
c. Architectural patterns / principles:
- Event‐driven messaging (decoupling producer and consumer)
- Single‐responsibility microservices (one service for produce, one for consume)
- Containerized infrastructure (Docker Compose for Kafka/Zookeeper)
- Spring Boot conventions (auto‐configuration, dependency injection)
d. Key technologies / frameworks:
- Apache Kafka (broker + Zookeeper)
- Spring Boot (producer & consumer apps)
- Spring Kafka (KafkaTemplate, @KafkaListener)
- Spring Data JPA (Consumer → MySQL)
- Maven multimodule build
- Docker Compose

## Diagram (Mermaid)

```mermaid
flowchart TD
    %% External Event Source
    SSE["Wikimedia SSE Feed"]:::external

    %% Producer Service
    Producer["Producer Service\n(Spring Boot)"]:::service

    %% Messaging Infrastructure
    subgraph "Messaging Infrastructure"
        direction TB
        KafkaBroker["Kafka Broker"]:::infra
        Zookeeper["Zookeeper"]:::infra
        Topic["Topic: wikievents"]:::infra
    end

    %% Consumer Service
    Consumer["Consumer Service\n(Spring Boot)"]:::service

    %% Database
    MySQL["MySQL Database"]:::external

    %% Parent Module & Infrastructure Files
    Pom["Parent Maven Module\n(pom.xml)"]:::infra
    DockerCompose["docker-compose.yml"]:::infra

    %% Connections
    SSE -->|"HTTP SSE / REST Stream"| Producer
    Producer -->|"publish to topic"| Topic
    Topic -->|"subscribe"| Consumer
    Consumer -->|"INSERT (JDBC)"| MySQL
    DockerCompose --> KafkaBroker
    DockerCompose --> Zookeeper
    Pom --> Producer
    Pom --> Consumer

    %% Click Events - Parent Module & Infra
    click Pom "https://github.com/choudharylakshyaveer/wikievent-stream-kafka/blob/master/pom.xml"
    click DockerCompose "https://github.com/choudharylakshyaveer/wikievent-stream-kafka/blob/master/docker-compose.yml"

    %% Click Events - Producer Service
    click Producer "https://github.com/choudharylakshyaveer/wikievent-stream-kafka/blob/master/wikievent-producer/pom.xml"
    click Producer "https://github.com/choudharylakshyaveer/wikievent-stream-kafka/blob/master/wikievent-producer/src/main/java/com/spring/java/wikievent/ProducerApplication.java"
    click Producer "https://github.com/choudharylakshyaveer/wikievent-stream-kafka/blob/master/wikievent-producer/src/main/java/com/spring/java/wikievent/KafkaTopicConfig.java"
    click Producer "https://github.com/choudharylakshyaveer/wikievent-stream-kafka/blob/master/wikievent-producer/src/main/java/com/spring/java/wikievent/WikimediaChangesHandler.java"
    click Producer "https://github.com/choudharylakshyaveer/wikievent-stream-kafka/blob/master/wikievent-producer/src/main/java/com/spring/java/wikievent/WikimediaChangesProducer.java"
    click Producer "https://github.com/choudharylakshyaveer/wikievent-stream-kafka/blob/master/wikievent-producer/src/main/resources/application.properties"

    %% Click Events - Consumer Service
    click Consumer "https://github.com/choudharylakshyaveer/wikievent-stream-kafka/blob/master/wikievent-consumer/pom.xml"
    click Consumer "https://github.com/choudharylakshyaveer/wikievent-stream-kafka/blob/master/wikievent-consumer/src/main/java/com/spring/java/wikievent/ConsumerApplication.java"
    click Consumer "https://github.com/choudharylakshyaveer/wikievent-stream-kafka/blob/master/wikievent-consumer/src/main/java/com/spring/java/wikievent/KafkaDatabaseConsumer.java"
    click Consumer "https://github.com/choudharylakshyaveer/wikievent-stream-kafka/blob/master/wikievent-consumer/src/main/java/com/spring/java/wikievent/entity/WikimediaData.java"
    click Consumer "https://github.com/choudharylakshyaveer/wikievent-stream-kafka/blob/master/wikievent-consumer/src/main/java/com/spring/java/wikievent/repository/WikimediaDataRepository.java"
    click Consumer "https://github.com/choudharylakshyaveer/wikievent-stream-kafka/blob/master/wikievent-consumer/src/main/resources/application.properties"

    %% Styles
    classDef external fill:#blue,stroke:#000,color:#fff;
    classDef infra fill:#f90,stroke:#000,color:#000;
    classDef service fill:#0f0,stroke:#000,color:#000;
```
