CODEWITHSUNDEEP
apache-kafkaapache-kafka-connectdebezium
Enoch Chu
Enoch Chu

Reputation: 61

Using Amazon MSK and Debezium SQL Server Connector. Error while fetching metadata with correlation id 7 : {TestKafkaDB= UNKNOWN_TOPIC_OR_PARTITION}

I was trying to connect my RDS MS SQL server with Debezium SQL Server Connector to stream changes to Kafka Cluster on Amazon MSK.

I configured connector and Kafka Connect worker well run the Connect by bin/connect-standalone.sh ../worker.properties connect/dbzmmssql.properties

Got WARN [Producer clientId=producer-1] Error while fetching metadata with correlation id 10 : {TestKafkaDB=UNKNOWN_TOPIC_OR_PARTITION} (org.apache.kafka.clients.NetworkClient:1031)

Upvotes: 2

Views: 1695

Answers (1)

Enoch Chu
Enoch Chu

Reputation: 61

I've solved this problem and just want to share my possible solution with other fresher with Kafka.

TestKafkaDB=UNKNOWN_TOPIC_OR_PARTITION basically means the connector didn't find the a usable topic in Kafka broker. The reason I am facing this is the Kafka broker didn't automatically create a new topic for the stream.

To solving this, I changed Cluster Configuration in AWS MSK console, change auto.create.topics.enable from default false to true and update this configuration to the Cluster, then my problem solved.

Upvotes: 3

Related Questions