CODEWITHSUNDEEP
postgresqlapache-kafkadebeziumcdc
Hans
Hans

Reputation: 31

Kafka Connect with Debezium connection to Azure Postgres error when plugin.name is wal2json

I have setup Kafka connect which starts fine. I'm trying to setup the Debezium source connector to an Azure Flexible Server using the wal2json shared library. I have enabled wal2json and can manually create a replication slot of type wal2json, but when I try to run:

curl -X POST -H "Content-Type: application/json" --data @pg-source-connector.json http://localhost:8083/connectors

I get the following error:

"error_code":400,"message":"Connector configuration is invalid and contains the following 1 error(s):The 'plugin.name' value is invalid: Value must be one of decoderbufs, pgoutput"

My config file has "plugin.name": "wal2json", is there any reason the Kafka connect would not accept the wal2json plugin? The Postgres database is configured to support it, I would have assumed it would read which plugins are supported by the database. Is there anywhere else on the Kafka side I need to enable this plugin?

I tried to manually create the replication slot on the database of type wal2json and it worked fine. This tells me the database will support it, but that Kafka is not seeing it as a available plugin. The Debezium documentation shows that the following are supported: "Supported values are decoderbufs, wal2json, wal2json_rds, wal2json_streaming, wal2json_rds_streaming and pgoutput."

Upvotes: 0

Views: 282

Answers (1)

manjunath
manjunath

Reputation: 306

If your using debezium version 2.x,it does not support wal2json. Debezium version 1.x supports it.

Since postgres community stopped support for wal2json,so in debezium 2.x they choosed to remove wal2json from debezium so that overall maintenance cost of connector is reduced.

Upvotes: 0

Related Questions