# sarthak1/kafka-pain-areas architecture

> Architecture diagram of the GitHub repository sarthak1/kafka-pain-areas, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-09-09). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/sarthak1/kafka-pain-areas
- Repository: https://github.com/sarthak1/kafka-pain-areas

## Overview

##### Project Type and Purpose
- This is a backend, event-driven Spring Boot microservice focused on two pain areas in a Kafka ecosystem:
– Real-time handling of reverse movements (Store → DC).
– One-time “cut-over” processing of historical data via a REST API plus Kafka.
- It is not full “frontend” + “backend” in the UI sense; it’s a single service with:
– Kafka inbound listener
– Business logic modules
– Database persistence
– External REST API integration for cut-over
– Unit tests

##### Key Architectural Components
A. Ingestion Layer
– KafkaListenerService
- Subscribes to one or more Kafka topics.
- At startup, can be paused during cut-over.
B. Cut-over Startup Task
– CutoverDataService
- Triggered on app start if `system.cutover.enabled=true`.
- Fetches historical data via SA-Team REST API.
- (Optionally) reads from Kafka for gap days.
C. Processing Layer (Business Logic)
– MovementProcessor
- Core reverse/normal movement detection.
- Determines MovementType (NORMAL/REVERSE).
- Maps actualOrigin/actualDestination.
– LocationService
- Identifies location types (Store/DC).
- Caching, prefix‐matching, pattern rules.
– MovementValidationService
- Validates processed movements against business rules.
D. Persistence Layer
– MovementDataRepository
- Spring Data JPA repository.
- Persists MovementData entities to a relational database (PostgreSQL/MySQL).
E. Infrastructure & Configuration
– Kafka cluster (external)
– SA Team REST API (external)
– Relational DB (PostgreSQL/MySQL)
– application.properties & KafkaPainAreasConfig.java

##### Interactions & Data Flow
1. Application Startup
– Check `system.cutover.enabled`.
– If true:
· Pause KafkaListenerService
· Call CutoverDataService.fetchAndProcessHistorical()
· Persist historical data (MovementData)
· Resume KafkaListenerService
2. Real-time Kafka Processing
– KafkaListenerService receives KafkaMovementMessage
– Delegates to MovementProcessor.processMovement()
– MovementProcessor calls LocationService → MovementValidationService
– On success, MovementDataRepository.save(…)
3. External Dependencies
– Kafka cluster for real-time messages
– SA Team REST API for cut-over data
– Relational DB for persisted state

##### Architectural Patterns & Principles
- Event-Driven Architecture (EDA) using Kafka as the message bus
- Single Application Bounded Context (no microservices split)
- Layered architecture:
– Ingestion → Processing → Persistence
- “Startup job” pattern for cut-over
- Dependency Injection (Spring Boot) for wiring services

## Diagram (Mermaid)

```mermaid
flowchart TB
    %% External Systems Layer
    subgraph "External Systems"
        direction TB
        KafkaCluster["Kafka Cluster"]:::external
        SATeamRESTAPI["SA Team REST API"]:::external
    end

    %% Application Modules Layer
    subgraph "Application Modules"
        direction TB
        Start(("Start")):::external
        CutoverDataService["CutoverDataService\n(startup job)"]:::ingestion
        KafkaListenerService["KafkaListenerService"]:::ingestion
        MovementProcessor["MovementProcessor"]:::processing
        subgraph "Helpers"
            direction TB
            LocationService["LocationService"]:::processing
            MovementValidationService["MovementValidationService"]:::processing
        end
        MovementDataRepository["MovementDataRepository"]:::persistence
        Config["KafkaPainAreasConfig\n+ application.properties"]:::external
        Tests["MovementProcessorTest"]:::external
    end

    %% Database Layer
    subgraph "Database"
        direction TB
        MovementData["MovementData\n(JPA Entity)"]:::persistenceDB
    end

    %% Connections
    KafkaCluster -->|"consume Topic=sa-team-topic"| KafkaListenerService
    KafkaListenerService -->|"processMovement()"| MovementProcessor
    MovementProcessor -->|"identifyType()"| LocationService
    MovementProcessor -->|"validate()"| MovementValidationService
    MovementProcessor -->|"save MovementData"| MovementDataRepository
    MovementDataRepository --> MovementData

    Start -->|"system.cutover.enabled?"| CutoverDataService
    CutoverDataService -->|"REST: GET /movements"| SATeamRESTAPI
    CutoverDataService -->|"save MovementData"| MovementDataRepository

    Start -->|"pause/resume"| KafkaListenerService

    %% Click Events
    click KafkaListenerService "https://github.com/sarthak1/kafka-pain-areas/blob/main/KafkaListenerService.java"
    click CutoverDataService "https://github.com/sarthak1/kafka-pain-areas/blob/main/CutoverDataService.java"
    click MovementProcessor "https://github.com/sarthak1/kafka-pain-areas/blob/main/MovementProcessor.java"
    click LocationService "https://github.com/sarthak1/kafka-pain-areas/blob/main/LocationService.java"
    click MovementValidationService "https://github.com/sarthak1/kafka-pain-areas/blob/main/MovementValidationService.java"
    click MovementDataRepository "https://github.com/sarthak1/kafka-pain-areas/blob/main/MovementDataRepository.java"
    click MovementData "https://github.com/sarthak1/kafka-pain-areas/blob/main/MovementData.java"
    click Config "https://github.com/sarthak1/kafka-pain-areas/blob/main/KafkaPainAreasConfig.java"
    click Config "https://github.com/sarthak1/kafka-pain-areas/blob/main/application.properties"
    click Tests "https://github.com/sarthak1/kafka-pain-areas/blob/main/MovementProcessorTest.java"

    %% Styles
    classDef external fill:#DDDDDD,stroke:#888
    classDef ingestion fill:#ADD8E6,stroke:#0366d6
    classDef processing fill:#90EE90,stroke:#228B22
    classDef persistence fill:#FFA500,stroke:#CC5500
    classDef persistenceDB fill:#FFA500,stroke:#CC5500,shape:database
```
