# waffleliew/mrt-crowd-tracker architecture

> Architecture diagram of the GitHub repository waffleliew/mrt-crowd-tracker, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-05-25). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/waffleliew/mrt-crowd-tracker
- Repository: https://github.com/waffleliew/mrt-crowd-tracker

## Overview

##### Project Type and Purpose
- This is a full-stack, real-time crowd-density tracking application for MRT stations.
- Frontend: Next.js map UI that displays live crowd data.
- Backend: Spring Boot service that fetches LTA DataMall API, processes via Kafka, stores in MongoDB, and serves data to the frontend.
- Infrastructure: Kafka (with Zookeeper), MongoDB (with Mongo-Express UI), all orchestrated via Docker Compose.

##### Main Components
a. External Data Source
- LTA DataMall REST API
b. Backend Service (Spring Boot)
- Kafka Producer (fetches from LTA, pushes to Kafka topic)
- Kafka Consumer (reads topic, processes messages)
- MongoConfig and Mongo Repository (stores processed crowd data)
- WebSocketEventListener (pushes real-time updates to frontend via WebSocket)
- REST Controllers (optional: serves on-demand crowd data via HTTP API)
c. Messaging Layer
- Zookeeper
- Kafka Broker (topic: crowd-density-updates)
d. Data Storage
- MongoDB
- Mongo-Express (Admin UI)
e. Frontend (Next.js)
- UI component (reads from WebSocket or REST endpoint)
- Static CSV data source for station locations (library/mrt_lrt_data.csv)
f. Container Orchestration
- Docker Compose network grouping all the above services

##### Relationships and Data Flows
1. Spring Boot “KafkaProducerService” polls LTA DataMall → produces JSON messages to Kafka topic.
2. Spring Boot “KafkaConsumerService” subscribes to the same topic → processes and writes records to MongoDB via “CrowdDensityRepo.”
3. WebSocketEventListener emits updates downstream to any connected frontend clients.
4. Frontend connects via:
– WebSocket (real-time push)
– Or REST GET /crowddensity (initial load or fallback)
5. Mongo-Express connects to MongoDB for admin UI on port 8081.

##### Architectural Patterns and Design Principles
- Event-Driven Architecture (Kafka pub/sub)
- Single Page Application (Next.js)
- Containerization (Docker, Docker Compose)
- Separation of concerns:
- Data Ingestion (producer)
- Data Processing (consumer, repository)
- Data Presentation (frontend)
- Infrastructure as code (docker-compose.yaml)

## Diagram (Mermaid)

```mermaid
flowchart TB
    %% External API
    LTA["LTA DataMall API"]:::external

    subgraph "Docker Compose Network"
        direction TB

        subgraph "Backend & Infrastructure"
            direction TB

            subgraph "Messaging Layer"
                direction TB
                Zookeeper["Zookeeper:2181"]:::messaging
                Kafka["Kafka Broker:9092"]:::messaging
            end

            subgraph "MongoDB Stack"
                direction TB
                MongoDB["MongoDB:27017"]:::database
                MongoExpress["Mongo-Express UI:8081"]:::application
            end

            subgraph "Spring Boot Service"
                direction TB
                DemoApp["DemoApplication"]:::application
                SB_Dockerfile["SpringBoot Dockerfile"]:::infra
                Producer["KafkaProducerService"]:::application
                ProducerConfig["KafkaProducerConfig"]:::application
                ConsumerConfig["KafkaConsumerConfig"]:::application
                Consumer["KafkaConsumerService"]:::application
                MapStream["MapStream"]:::application
                WSListener["WebSocketEventListener"]:::application
                MongoConfig["MongoConfig"]:::application
                Repo["CrowdDensityRepo"]:::application
                Model["CrowdDensity"]:::application
                Controller["CrowdDensityController"]:::application
            end
        end

        subgraph "Frontend"
            direction TB
            FE_Dockerfile["Frontend Dockerfile"]:::infra
            AppWrapper["_app.tsx"]:::application
            IndexPage["index.tsx (Map UI)"]:::application
            LocationsUtil["locations.tsx"]:::application
            CSV["mrt_lrt_data.csv"]:::external
        end
    end

    %% Data Flows
    LTA -->|polls REST| Producer
    Producer -->|produces to topic| Kafka
    Kafka -->|consumed by| Consumer
    Consumer -->|processes via| MapStream
    MapStream -->|writes via repo| Repo
    Repo -->|stores| MongoDB
    MongoDB -->|admin UI| MongoExpress
    Consumer -->|triggers| WSListener
    WSListener -->|pushes via WebSocket| IndexPage
    IndexPage -->|initial GET /crowddensity| Controller
    Controller -->|reads from repo| Repo
    CSV -->|loads static data| IndexPage

    %% Click Events
    click Producer "https://github.com/waffleliew/mrt-crowd-tracker/blob/main/SpringBoot/src/main/java/com/example/demo/kafka/Producer/KafkaProducerService.java"
    click ProducerConfig "https://github.com/waffleliew/mrt-crowd-tracker/blob/main/SpringBoot/src/main/java/com/example/demo/kafka/Producer/KafkaProducerConfig.java"
    click Consumer "https://github.com/waffleliew/mrt-crowd-tracker/blob/main/SpringBoot/src/main/java/com/example/demo/kafka/Consumer/KafkaConsumerService.java"
    click ConsumerConfig "https://github.com/waffleliew/mrt-crowd-tracker/blob/main/SpringBoot/src/main/java/com/example/demo/kafka/Consumer/KafkaConsumerConfig.java"
    click MapStream "https://github.com/waffleliew/mrt-crowd-tracker/blob/main/SpringBoot/src/main/java/com/example/demo/kafka/Consumer/MapStream.java"
    click WSListener "https://github.com/waffleliew/mrt-crowd-tracker/blob/main/SpringBoot/src/main/java/com/example/demo/kafka/Consumer/WebSocketEventListener.java"
    click MongoConfig "https://github.com/waffleliew/mrt-crowd-tracker/blob/main/SpringBoot/src/main/java/com/example/demo/config/MongoConfig.java"
    click Repo "https://github.com/waffleliew/mrt-crowd-tracker/blob/main/SpringBoot/src/main/java/com/example/demo/repository/CrowdDensityRepo.java"
    click Model "https://github.com/waffleliew/mrt-crowd-tracker/blob/main/SpringBoot/src/main/java/com/example/demo/model/CrowdDensity.java"
    click Controller "https://github.com/waffleliew/mrt-crowd-tracker/blob/main/SpringBoot/src/main/java/com/example/demo/controller/CrowdDensityController.java"
    click DemoApp "https://github.com/waffleliew/mrt-crowd-tracker/blob/main/SpringBoot/src/main/java/com/example/demo/DemoApplication.java"
    click SB_Dockerfile "https://github.com/waffleliew/mrt-crowd-tracker/tree/main/SpringBoot/Dockerfile"
    click FE_Dockerfile "https://github.com/waffleliew/mrt-crowd-tracker/tree/main/Frontend/Dockerfile"
    click AppWrapper "https://github.com/waffleliew/mrt-crowd-tracker/blob/main/Frontend/pages/_app.tsx"
    click IndexPage "https://github.com/waffleliew/mrt-crowd-tracker/blob/main/Frontend/pages/index.tsx"
    click LocationsUtil "https://github.com/waffleliew/mrt-crowd-tracker/blob/main/Frontend/library/locations.tsx"
    click CSV "https://github.com/waffleliew/mrt-crowd-tracker/blob/main/Frontend/library/mrt_lrt_data.csv"
    click docker-compose "https://github.com/waffleliew/mrt-crowd-tracker/blob/main/docker-compose.yaml"

    %% Styles
    classDef application fill:#D0E8FF,stroke:#0066CC,stroke-width:1px
    classDef messaging fill:#FFE4B3,stroke:#CC8400,stroke-width:1px
    classDef database fill:#DAF7DC,stroke:#28A745,stroke-width:1px
    classDef external fill:#F0F0F0,stroke:#888888,stroke-width:1px
    classDef infra fill:#E8D0FF,stroke:#6A0DAD,stroke-width:1px
```
