How trendyol/go-pq-cdc works
Main Components
a. Core Components:
• PostgreSQL Database (Source)
• go-pq-cdc Connector
• Downstream Systems (Kafka, Elasticsearch, etc.)
• Monitoring Stack (Prometheus/Grafana)
b. Internal Components of go-pq-cdc:
• Configuration Manager
• Replication Manager
• Publication Manager
• Message Handler
• Metrics Collector
• HTTP Server
[Source Layer]
• PostgreSQL Database
• ↓
[CDC Layer (go-pq-cdc)]
• Connector Core
• ↓
[Destination Layer]
• Downstream Systems
[Configuration Manager] → [Connector Core] ← [HTTP Server]
• ↑ ↓
• Config Files [Replication Manager]
• ↓
• [Publication Manager]
• ↓
• [Message Handler]
• ↓
• [Metrics Collector]
Key Interactions to Highlight
a. Replication Flow:
• PostgreSQL WAL → Replication Slot → Message Handler
• Message Format Processing (Insert/Update/Delete)
• Message Delivery to Handler Function
b. Management Flow:
• Configuration Loading
• Metrics Collection
• Health Checking
• Slot Activity Monitoring
Important Technical Details
• Replication Slot Management
• Publication Configuration
• Message Format Types
• Metric Endpoints
• Health Check API
Specific Implementation Notes
• The passive/active availability model
• Logical replication capabilities
• Metric collection points
• Configuration injection points
• Error handling paths
Component Descriptions
• Component roles and responsibilities
• Key configuration parameters
• Monitoring points
• Integration interfaces