CODEWITHSUNDEEP
compressiongoogle-cloud-pubsubdebeziumdebezium-server
Divya Dass
Divya Dass

Reputation: 166

Payload compression at Debezium Server. "Debezium Server - Pub/Sub Pipeline"

We have a setup where we are using Debezium to read the logs from MYSQL, sending them across to Pub/Sub, and then transferring them to BigQuery via BQ Subscriptions. This pipeline is working fine but the cost is more than expected (at Pub/Sub) as the CDC data volume is rather high (~1.5 TB/day)

We are trying to see if we can compress the payload at Debezium server itself before we send it across to Pub/Sub. We are not seeing any properties specific to compression on the Debezium server documentation page. Can anyone help out here?

Upvotes: 0

Views: 84

Answers (0)

Related Questions