How alibaba/canal works
Canal captures MySQL changes by acting as a replica, parses binlog events, and makes them available for client consumption; adapter modules can transform those changes into target stores. The repository also includes Canal Admin for web-based configuration and operational management, plus Prometheus monitoring. The README documents the core capture behavior and major capabilities; sampled adapter code shows DML synchronization and ETL behavior. Admin wiring and several runtime links are not shown in the supplied excerpts, so those connections are marked unverified.
Main entry points: Dml.java Change records · RdbSyncService.java Target sync services · MysqlQueryExecutor.java MySQL protocol driver · ESSyncService.java Elasticsearch sync
Components 25
Binlog capture
- MySQL protocol driver
driver/src/main/java/com/alibaba/otter/canal/parse/driver/mysql/MysqlQueryExecutor.java - Binlog parser
parse/src/main/java/com/alibaba/otter/canal/parse/ha/HeartBeatHAController.java - Event sink
sink/src/main/java/com/alibaba/otter/canal/sink/CanalEventDownStreamHandler.java
Event delivery
- Canal server
server/src/main/java/com/alibaba/otter/canal/server/CanalService.java - Client protocol
protocol/src/main/java/com/alibaba/otter/canal/protocol/ClientIdentity.java - Canal client
client/src/main/java/com/alibaba/otter/canal/client/CanalConnector.java
Client adapters
- Adapter launcher
client-adapter/launcher/src/main/java/com/alibaba/otter/canal/adapter/launcher/loader/CanalAdapterService.java - Adapter configuration
client-adapter/common/src/main/java/com/alibaba/otter/canal/client/adapter/support/AdapterConfig.java - Change records
client-adapter/common/src/main/java/com/alibaba/otter/canal/client/adapter/support/Dml.java - Target sync services
client-adapter/rdb/src/main/java/com/alibaba/otter/canal/client/adapter/rdb/service/RdbSyncService.java - Full-load ETL
client-adapter/common/src/main/java/com/alibaba/otter/canal/client/adapter/support/AbstractEtlService.java - Elasticsearch sync
client-adapter/escore/src/main/java/com/alibaba/otter/canal/client/adapter/es/core/service/ESSyncService.java - HBase sync
client-adapter/hbase/src/main/java/com/alibaba/otter/canal/client/adapter/hbase/service/HbaseSyncService.java - Phoenix sync
client-adapter/phoenix/src/main/java/com/alibaba/otter/canal/client/adapter/phoenix/service/PhoenixSyncService.java - ClickHouse sync
client-adapter/clickhouse/src/main/java/com/alibaba/otter/canal/client/adapter/clickhouse/service/ClickHouseBatchSyncService.java
Operations and management
- Admin web UI
admin/admin-ui/src/views/canalServer/CanalInstance.vue - Admin API
admin/admin-web/src/main/java/com/alibaba/otter/canal/admin/controller/CanalInstanceController.java - Admin services
admin/admin-web/src/main/java/com/alibaba/otter/canal/admin/service/impl/CanalInstanceServiceImpl.java - Admin database
admin/admin-web/src/main/java/com/alibaba/otter/canal/admin/config/EbeanConfig.java - Prometheus metrics
prometheus/src/main/java/com/alibaba/otter/canal/prometheus/PrometheusService.java
Other components
- MySQL master
- Consumer application
- Target data store
- Canal operator
- Metrics scraper
Connections 21
- MySQL protocol driver → MySQL master: requests binlog
README.md - MySQL master → MySQL protocol driver: streams events
README.md - MySQL protocol driver → Binlog parser: passes byte stream
- Binlog parser → Event sink: forwards events
- Canal server → Consumer application: serves subscriptions
- Consumer application → Canal client: uses connector
- Adapter launcher → Adapter configuration: loads settings
- Target sync services → Change records: reads changes
client-adapter/rdb/src/main/java/com/alibaba/otter/canal/client/adapter/rdb/service/RdbSyncService.java - Target sync services → Adapter configuration: uses mappings
client-adapter/rdb/src/main/java/com/alibaba/otter/canal/client/adapter/rdb/service/RdbSyncService.java - Elasticsearch sync → Change records: reads changes
client-adapter/escore/src/main/java/com/alibaba/otter/canal/client/adapter/es/core/service/ESSyncService.java - Elasticsearch sync → Target data store: writes indexes
- HBase sync → Change records: reads changes
client-adapter/hbase/src/main/java/com/alibaba/otter/canal/client/adapter/hbase/service/HbaseSyncService.java - Phoenix sync → Change records: reads changes
client-adapter/phoenix/src/main/java/com/alibaba/otter/canal/client/adapter/phoenix/service/PhoenixSyncService.java - ClickHouse sync → Change records: reads changes
client-adapter/clickhouse/src/main/java/com/alibaba/otter/canal/client/adapter/clickhouse/service/ClickHouseBatchSyncService.java - Full-load ETL → Target data store: imports full data
- Target sync services → Target data store: applies changes
- Canal operator → Admin web UI: manages instances
- Admin web UI → Admin API: sends requests
- Admin API → Admin services: delegates
- Admin services → Admin database: persists settings
- Metrics scraper → Prometheus metrics: scrapes metrics