How kushalkhadka7/streaming-cdc works
This repository is described as a PostgreSQL/MSSQL change-data-capture pipeline: producers write database changes, Debezium publishes table-specific events to Kafka, and Flink processes them for Kafka or Elasticsearch sinks. The README also names the schema registry, Kafka Control Center, and product/order load generators. The supplied tree contains only installed Python packages, not the application implementation, so paths are null and edges reflect documented flow rather than inspected calls.
Components 15
Data Sources
- Product Loader
- Order Loader
- PostgreSQL
- MSSQL
Change Capture
- Debezium
- Kafka Cluster
- Table Topics
- Schema Registry
Stream Processing
- Flink Cluster
- Flink Job
- Kafka Sink
- Elasticsearch Sink
Operations
- Kafka Control Center
- Flink Dashboard
Other components
- Writing Services
Connections 15
- Writing Services → PostgreSQL: writes changes
- Writing Services → MSSQL: writes changes
- Product Loader → MSSQL: loads products
- Order Loader → PostgreSQL: writes orders
- PostgreSQL → Debezium: captures changes
- MSSQL → Debezium: captures changes
- Debezium → Table Topics: publishes events
- Table Topics → Kafka Cluster: stored in
- Debezium → Schema Registry: uses schemas
- Flink Job → Kafka Cluster: reads streams
- Flink Cluster → Flink Job: runs job
- Flink Job → Kafka Sink: sends processed data
- Flink Job → Elasticsearch Sink: writes processed data
- Kafka Control Center → Kafka Cluster: monitors cluster
- Flink Dashboard → Flink Cluster: shows status