# confluentinc/cp-demo architecture

> Architecture diagram of the GitHub repository confluentinc/cp-demo, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-09-28). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/confluentinc/cp-demo
- Repository: https://github.com/confluentinc/cp-demo

## Overview

This repository is a deployable Kafka event-streaming demo for live Wikipedia edits. The principal workflow ingests Wikimedia events, transforms and processes them with ksqlDB and Kafka Streams, then indexes results in Elasticsearch for Kibana analysis. The README documents the end-to-end flow; the sampled Java source confirms the Kafka Streams stage and its input/output topics. Other product stages and monitoring/security capabilities are documented, but their implementation wiring was not sampled, so those links follow the README only.

Main entry points: [`kstreams-app/src/main/java/io/confluent/demos/common/wiki/WikipediaActivityMonitor.java`](https://github.com/confluentinc/cp-demo/blob/8.1.0-post/kstreams-app/src/main/java/io/confluent/demos/common/wiki/WikipediaActivityMonitor.java) (Activity monitor), [`scripts/security`](https://github.com/confluentinc/cp-demo/tree/8.1.0-post/scripts/security) (Security configuration), [`scripts/connectors/submit_wikipedia_sse_config.sh`](https://github.com/confluentinc/cp-demo/blob/8.1.0-post/scripts/connectors/submit_wikipedia_sse_config.sh) (SSE source setup), [`scripts/connectors/submit_elastic_sink_config.sh`](https://github.com/confluentinc/cp-demo/blob/8.1.0-post/scripts/connectors/submit_elastic_sink_config.sh) (Elasticsearch sink setup)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_ingest["Event ingestion"]
  node_source_config["SSE source setup"]
  node_connect["Kafka Connect"]
  node_transform["JSON Schema transform"]
  node_kafka[("Kafka cluster")]
end

subgraph group_process["Stream processing"]
  node_ksql["ksqlDB processing<br/>[statements.sql]"]
  node_activity["Activity monitor"]
  node_avro["Avro event schemas"]
end

subgraph group_analysis["Search and analysis"]
  node_sink_config["Elasticsearch sink setup"]
  node_sink["Elasticsearch sink connector"]
  node_elasticsearch[("Elasticsearch")]
  node_kibana["Kibana dashboards"]
end

subgraph group_platform["Platform operations"]
  node_schema[("Schema Registry")]
  node_control["Control Center"]
  node_security["Security configuration"]
end

node_wiki(("Wikimedia EventStreams"))
node_operator(("Operator"))

node_wiki -->|"publishes events"| node_connect
node_operator -->|"submits setup"| node_source_config
node_source_config -->|"configures source"| node_connect
node_connect -->|"transforms events"| node_transform
node_transform -->|"writes Avro"| node_kafka
node_schema -->|"provides schemas"| node_connect
node_kafka -->|"streams records"| node_ksql
node_kafka -->|"reads parsed topic"| node_activity
node_activity -->|"writes domain counts"| node_kafka
node_avro -->|"defines metric type"| node_activity
node_operator -->|"submits setup"| node_sink_config
node_sink_config -->|"configures sink"| node_sink
node_kafka -->|"streams records"| node_sink
node_sink -->|"indexes events"| node_elasticsearch
node_elasticsearch -->|"serves analysis"| node_kibana
node_schema -->|"provides schemas"| node_activity
node_control -.->|"manages and monitors"| node_kafka
node_security -.->|"secures access"| node_kafka
node_security -.->|"secures access"| node_connect

click node_source_config "https://github.com/confluentinc/cp-demo/blob/8.1.0-post/scripts/connectors/submit_wikipedia_sse_config.sh"
click node_ksql "https://github.com/confluentinc/cp-demo/blob/8.1.0-post/scripts/ksqlDB/statements.sql"
click node_activity "https://github.com/confluentinc/cp-demo/blob/8.1.0-post/kstreams-app/src/main/java/io/confluent/demos/common/wiki/WikipediaActivityMonitor.java"
click node_avro "https://github.com/confluentinc/cp-demo/tree/8.1.0-post/kstreams-app/src/main/avro"
click node_sink_config "https://github.com/confluentinc/cp-demo/blob/8.1.0-post/scripts/connectors/submit_elastic_sink_config.sh"
click node_kibana "https://github.com/confluentinc/cp-demo/tree/8.1.0-post/scripts/dashboard"
click node_security "https://github.com/confluentinc/cp-demo/tree/8.1.0-post/scripts/security"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_source_config,node_connect,node_transform,node_kafka toneBlue
class node_ksql,node_activity,node_avro toneAmber
class node_sink_config,node_sink,node_elasticsearch,node_kibana toneMint
class node_schema,node_control,node_security toneRose
class node_wiki,node_operator toneIndigo
```

## Components

### Event ingestion

- [SSE source setup](https://github.com/confluentinc/cp-demo/blob/8.1.0-post/scripts/connectors/submit_wikipedia_sse_config.sh): `scripts/connectors/submit_wikipedia_sse_config.sh`
- **Kafka Connect**
- **JSON Schema transform**
- **Kafka cluster**

### Stream processing

- [ksqlDB processing](https://github.com/confluentinc/cp-demo/blob/8.1.0-post/scripts/ksqlDB/statements.sql): `scripts/ksqlDB/statements.sql`
- [Activity monitor](https://github.com/confluentinc/cp-demo/blob/8.1.0-post/kstreams-app/src/main/java/io/confluent/demos/common/wiki/WikipediaActivityMonitor.java): `kstreams-app/src/main/java/io/confluent/demos/common/wiki/WikipediaActivityMonitor.java`
- [Avro event schemas](https://github.com/confluentinc/cp-demo/tree/8.1.0-post/kstreams-app/src/main/avro): `kstreams-app/src/main/avro`

### Search and analysis

- [Elasticsearch sink setup](https://github.com/confluentinc/cp-demo/blob/8.1.0-post/scripts/connectors/submit_elastic_sink_config.sh): `scripts/connectors/submit_elastic_sink_config.sh`
- **Elasticsearch sink connector**
- **Elasticsearch**
- [Kibana dashboards](https://github.com/confluentinc/cp-demo/tree/8.1.0-post/scripts/dashboard): `scripts/dashboard`

### Platform operations

- **Schema Registry**
- **Control Center**
- [Security configuration](https://github.com/confluentinc/cp-demo/tree/8.1.0-post/scripts/security): `scripts/security`

### Other components

- **Wikimedia EventStreams**
- **Operator**

## Connections

- Wikimedia EventStreams → Kafka Connect: publishes events
- Operator → SSE source setup: submits setup
- SSE source setup → Kafka Connect: configures source
- Kafka Connect → JSON Schema transform: transforms events
- JSON Schema transform → Kafka cluster: writes Avro
- Schema Registry → Kafka Connect: provides schemas
- Kafka cluster → ksqlDB processing: streams records
- Kafka cluster → Activity monitor: reads parsed topic
- Activity monitor → Kafka cluster: writes domain counts
- Avro event schemas → Activity monitor: defines metric type
- Operator → Elasticsearch sink setup: submits setup
- Elasticsearch sink setup → Elasticsearch sink connector: configures sink
- Kafka cluster → Elasticsearch sink connector: streams records
- Elasticsearch sink connector → Elasticsearch: indexes events
- Elasticsearch → Kibana dashboards: serves analysis
- Schema Registry → Activity monitor: provides schemas
- Control Center → Kafka cluster: manages and monitors
- Security configuration → Kafka cluster: secures access
- Security configuration → Kafka Connect: secures access