CODEWITHSUNDEEP
oracle-databaseapache-kafkadebezium
Akshay Doifode
Akshay Doifode

Reputation: 1

Throughput issue on Target Kafka topic when using Oracle CDC confluent connector

We are currently planning to implement confluent oracle CDC connector and we are under testing phase. While testing we observed there is throughput issue on Target kafka topic. We tuned multiple parameters but the throughput is not going beyond 450Kbps. Anybody facing issue with confluent oracle cdc connector.

And we also performed testing on debezium oracle connector and we found it to be more stable than confluent oracle cdc connector.

Throughput on Target topic to be optimum

Upvotes: 0

Views: 356

Answers (1)

ibre5041
ibre5041

Reputation: 5298

From mine experience both tools face the same problem. They rely on Oracle Logminer - a diagnostic tool, which was intended for different purpose.

Check DB server, its performance and alert.log and you will find there is one spinning job running Logminer, utilizing 100% of single CPU all the time.

You might play with redolog size, try to add PGA memory(Logminer is memory hungry), but still you wont be able to scale up more. You might not be able to improve performance by starting multiple parallel Logminer jobs.

Try simulate a data purging job. Create huge tabla with some data. Then DELETE that data using single or more conventional DELETE statement(s). This will generate HUGE amount of redo. Check how backlog is growing and how fast can logminer parse generated arch redologs.

The only tools which can do native CDC for Oracle are: GoldenGate, Shareplex and Informatica.

Upvotes: 0

Related Questions