Reputation: 284
I am using debezium connector for mssql to push cdc events to kafka. There are millions of events to be pushed to kafka per minute but its lagging and is not able to keep it upto the speed of updations.
I am running connector in kafka's distributed mode. I assume since it is distributed, we can add more node to read binlog in parallel and increase performance but there is no such information on google how to do that.
I saw similar posts but everyone asked not to use multiple connectors for different set of tables for the same database as this doesn't improve the performance as both of them still will read the same serial binlog, specially when topic prefix is same.
If not, then whats the benefit of ditributed mode?
Upvotes: 0
Views: 486