How creaivelabs-test/kafka-payments-pipeline-benchmark works
This repository benchmarks a real-time payments pipeline, measuring throughput and latency across synthetic or IBM TabFormer transaction runs. The principal workflow is Generator → Kafka topics → Enricher → Scorer → Ledger → PostgreSQL; Redis supports enrichment lookups, and the Scorer maintains velocity state. The README documents these runtime stages and optional cluster/fault scenarios; the only sampled source is the IBM evaluation script, so other implementation links are documentation-backed rather than source-verified.
Main entry points: java Transaction enricher · ibm_eval.py IBM ledger evaluator
Components 12
Transaction Pipeline
- Transaction generator
src/main/java - Kafka topics
- Transaction enricher
src/main/java - Velocity scorer
src/main/java - Ledger writer
src/main/java
State and Storage
- Enrichment cache
- Payments database
- Scorer state store
Benchmark Evaluation
- IBM TabFormer data
- IBM ledger evaluator
scripts/ibm_eval.py - Fraud metrics
scripts/ibm_eval.py
Other components
- Benchmark operator
Connections 15
- Benchmark operator → Transaction generator: starts run
README.md - Transaction generator → Kafka topics: publishes transactions
README.md - Kafka topics → Transaction enricher: supplies transactions
README.md - Transaction enricher → Enrichment cache: checks cache
README.md - Transaction enricher → Payments database: loads cache misses
README.md - Transaction enricher → Kafka topics: publishes enriched
README.md - Kafka topics → Velocity scorer: supplies enriched
README.md - Velocity scorer → Scorer state store: stores velocity state
README.md - Velocity scorer → Kafka topics: publishes decisions
README.md - Kafka topics → Ledger writer: supplies decisions
README.md - Ledger writer → Payments database: writes ledger rows
README.md - Benchmark operator → IBM TabFormer data: selects replay data
README.md - IBM TabFormer data → IBM ledger evaluator: provides source rows
scripts/ibm_eval.py - IBM ledger evaluator → Payments database: reads ledger
scripts/ibm_eval.py - IBM ledger evaluator → Fraud metrics: writes metrics
scripts/ibm_eval.py