CODEWITHSUNDEEP
apache-kafkatransformationapache-kafka-connectdebeziumtombstone
Steven Chu
Steven Chu

Reputation: 91

Kafka Connect tombstone message doesn't apply InsertField transformation

InsertField transformation isn't applied in tombstone message.

I am using postgres 10.1 / debezium 1.2.1 version / kafka 2.4.1 version

for example.

{
...
    "transforms": "InsertField",
    "transforms.InsertField.type": "org.apache.kafka.connect.transforms.InsertField$Key",
    "transforms.InsertField.static.field": "__host",
    "transforms.InsertField.static.value": "abc.com",
...
}

key - {"id":1, "__host": "abc.com"}, value - {"id":1, "__host": "abc.com", "col1": 1, "col2": "a"}

key - {"id":1, "__host": "abc.com"} value - {"id":1, "__host": "abc.com", "col1": null, "col2": null}
key - {"id":1} value - null

Why is it happened? Is it bug? or Am I wrong?

Upvotes: 1

Views: 617

Answers (1)

Iskuskov Alexander
Iskuskov Alexander

Reputation: 4375

Expected behavior of InsertField transformation for tombstone messages:

The key field is inserted, and the value remains null.

But in your Kafka Connect version (2.4.1) there is a bug: the key field is not inserted. It was fixed in KAFKA-9707: InsertField.Key transformation should apply to tombstone records

You can upgrade your Kafka Connect cluster to some fix version (e.g. 2.4.2).

Upvotes: 1

Related Questions