CODEWITHSUNDEEP
apache-kafkaapache-kafka-connectconfluent-schema-registrydebezium
Vlad Gudikov
Vlad Gudikov

Reputation: 113

Changing the schema of the message published by Debezium Kafka Connect

I am currently working on setting up Debezium Connector for mysql using Outbox Event Router. I already have a working solution for message publishing without using Debezium and I need to configure connector that will fit the currently existing avro schema registered in confluent schema registry. The way debezium connector is configured right now it publishes both key and value in an array. The published message schema looks like this

["null",{<description of json blob stored in `message` column of outbox table>}]

Is there a way to publish just the value? My configuration looks like this

{
  "connector.class" : "io.debezium.connector.mysql.MySqlConnector",
  "tasks.max" : "1",
  "database.hostname" : "*****",
  "database.port" : "3306",
  "database.user" : "***",
  "database.password" : "***",
  "database.server.id" : "*****",
  "database.include.list" : "****",
  "table.include.list" : "****",
  "schema.history.internal.kafka.bootstrap.servers" : "*****",
  "schema.history.internal.kafka.topic" : "schemahistory.changelog",
  "transforms" : "outbox",
  "transforms.outbox.type" : "io.debezium.transforms.outbox.EventRouter",
  "transforms.outbox.table.expand.json.payload" : "true",
  "value.converter" : "io.debezium.converters.BinaryDataConverter",
  "value.converter.delegate.converter.type" : "io.confluent.connect.avro.AvroConverter",
  "transforms.outbox.table.field.event.key" : "message_key",
  "transforms.outbox.table.field.event.id" : "outbox_id",
  "transforms.outbox.route.topic.replacement" : "${routedByValue}",
  "transforms.outbox.route.by.field" : "topic",
  "transforms.outbox.table.field.event.payload" : "message",
  "topic.prefix" : "*****",
  "value.converter.delegate.converter.type.schema.registry.url" : "****",
  "value.converter.delegate.converter.type.basic.auth.credentials.source" : "USER_INFO",
  "value.converter.delegate.converter.type.basic.auth.user.info" : "*****:****",
  "value.converter.delegate.converter.type.schemas.enable" : "false",
  "value.converter.delegate.converter.type.auto.register.schemas" : "false",
  "key.converter.auto.register.schemas" : "false",
  "key.converter.schemas.enable" : "false",
  "name" : "message-publisher"
}

Is there a way to update the configuration so that message consists only from data stored in message column and nothing else, so that message published into topic looks like

{<description of json blob stored in `message` column of outbox table>}

Upvotes: 0

Views: 205

Answers (0)

Related Questions