How pranav-8660/website-clickstream-tracker works
Project Type and Purpose
• This is a microservices-based event-tracking pipeline.
• Purpose: collect frontend events, stream them through Kafka, enrich & process them, persist to multiple datastores, and surface analytics via an API.
File Structure
• Top-level directories each host a Spring Boot microservice:
• event-collector-service (ECS)
• event-processor-service (EPS)
• storage-service
• analytics-api-service
• You also have a docker-compose.yaml to orchestrate containers.
• Each service has its own pom.xml, src/main/java with controllers, services, DTOs, configs, and src/main/resources for application.properties.
• No frontend code here, but README calls out a separate JS SDK.
README for Insights
• It explicitly names your services and their responsibilities.
• Confirms use of Kafka topics (“clickstream-events”, “processed-events”), and three datastores (Postgres, MongoDB, Elasticsearch).
• Mentions an optional configuration service for the future.
Main Components and Data Flows
a. Frontend Tracker SDK
• A lightweight JS library (external to this repo) embedded in web pages.
• Sends events (beacon/batch) to ECS over HTTP.
b. Event Collector Service (ECS)
• Spring Boot REST API endpoint (/events).
• Produces raw events into Kafka topic “clickstream-events”.
c. Event Processor Service (EPS)
• Kafka consumer on “clickstream-events”.
• Enriches records (GeoIP, device parse, session stitching).
• Produces enriched events onto “processed-events” topic.
d. Storage Service
• Kafka consumer on “processed-events”.
• Persists to multiple datastores:
• PostgreSQL for relational/session querying
• MongoDB for flexible JSON storage
• Elasticsearch for search/dashboards
e. Analytics API Service (AAS)
• Reads from databases, provides ML, reporting & visualization endpoints.
f. (Future) Configuration Service
• REST API to supply tracking rules/config to the Frontend SDK.
Architectural Patterns & Design Principles
• Microservices, each packaged & deployed independently.
• Event-driven, publish/subscribe pipeline with Kafka.
• Single responsibility per service.
• Separation of ingestion, enrichment, storage, and analytics.
• Stateless services behind containers for easy horizontal scaling.
• Pluggable datastore layer.
Technologies & Frameworks
• Spring Boot (all Java services)
• Apache Kafka (messaging)
• PostgreSQL, MongoDB, Elasticsearch (storage)
• Docker Compose for local orchestration
• JS SDK (frontend, external)
• (Future) Config Server (Spring Cloud Config?)