How confluentinc/cp-demo works
This repository is a deployable Kafka event-streaming demo for live Wikipedia edits. The principal workflow ingests Wikimedia events, transforms and processes them with ksqlDB and Kafka Streams, then indexes results in Elasticsearch for Kibana analysis. The README documents the end-to-end flow; the sampled Java source confirms the Kafka Streams stage and its input/output topics. Other product stages and monitoring/security capabilities are documented, but their implementation wiring was not sampled, so those links follow the README only.
Main entry points: WikipediaActivityMonitor.java Activity monitor · security Security configuration · submit_wikipedia_sse_config.sh SSE source setup · submit_elastic_sink_config.sh Elasticsearch sink setup
Components 16
Event ingestion
- SSE source setup
scripts/connectors/submit_wikipedia_sse_config.sh - Kafka Connect
- JSON Schema transform
- Kafka cluster
Stream processing
- ksqlDB processing
scripts/ksqlDB/statements.sql - Activity monitor
kstreams-app/src/main/java/io/confluent/demos/common/wiki/WikipediaActivityMonitor.java - Avro event schemas
kstreams-app/src/main/avro
Search and analysis
- Elasticsearch sink setup
scripts/connectors/submit_elastic_sink_config.sh - Elasticsearch sink connector
- Elasticsearch
- Kibana dashboards
scripts/dashboard
Platform operations
- Schema Registry
- Control Center
- Security configuration
scripts/security
Other components
- Wikimedia EventStreams
- Operator
Connections 19
- Wikimedia EventStreams → Kafka Connect: publishes events
- Operator → SSE source setup: submits setup
- SSE source setup → Kafka Connect: configures source
- Kafka Connect → JSON Schema transform: transforms events
- JSON Schema transform → Kafka cluster: writes Avro
- Schema Registry → Kafka Connect: provides schemas
- Kafka cluster → ksqlDB processing: streams records
- Kafka cluster → Activity monitor: reads parsed topic
- Activity monitor → Kafka cluster: writes domain counts
- Avro event schemas → Activity monitor: defines metric type
- Operator → Elasticsearch sink setup: submits setup
- Elasticsearch sink setup → Elasticsearch sink connector: configures sink
- Kafka cluster → Elasticsearch sink connector: streams records
- Elasticsearch sink connector → Elasticsearch: indexes events
- Elasticsearch → Kibana dashboards: serves analysis
- Schema Registry → Activity monitor: provides schemas
- Control Center → Kafka cluster: manages and monitors
- Security configuration → Kafka cluster: secures access
- Security configuration → Kafka Connect: secures access