CODEWITHSUNDEEP
apache-kafkaapache-kafka-connectdebezium
ThreeDots
ThreeDots

Reputation: 767

Debezium, Kafka connect: is there a way to send only payload and not schema?

I have an outbox postgresql table and debezium connector in kafka connect that creates kafka messages based on the added rows to the table.

The problem I am facing is with the message format. This is the created message value:

{
  "schema": {
    "type": "string",
    "optional": true,
    "name": "io.debezium.data.Json",
    "version": 1
  },
  "payload": "{\"foo\": \"bar\"}"
}

But (because of consumer) I need the message to contain only the payload, like this:

{
  "\"foo\": \"bar\""
}

This is part of my kafka connector configuration:

"transforms": "outbox",
"transforms.outbox.type": "io.debezium.transforms.outbox.EventRouter",
"transforms.outbox.route.topic.replacement": "${routedByValue}",
"transforms.outbox.route.by.field": "aggregate_type",
"transforms.outbox.table.field.event.payload.id": "aggregate_id",
"transforms.outbox.table.fields.additional.placement": "payload_type:header:__TypeId__"

Is there any way to achieve this without creating custom transformer?

Upvotes: 4

Views: 4573

Answers (1)

Robin Moffatt
Robin Moffatt

Reputation: 32130

It looks like you're using org.apache.kafka.connect.json.JsonConverter with schemas.enable=true for your value converter. When you do this it embeds the schema alongside the payload in the message.

If you set value.converter.schemas.enable=false you should get just the payload in your message.

Ref: Kafka Connect: Converters and Serialization Explained — JSON and Schemas

Upvotes: 6

Related Questions