CODEWITHSUNDEEP
apache-kafkaserialization
Daniel P.
Daniel P.

Reputation: 369

Kafka Producer without key serializer

I have a kakfa producer where I do not need to serialize a key, just the values. But the producer config requires the "key.serializer" setting.

Map<String, Object> producerConfig = new HashMap<>();
producerConfig.put(ProducerConfig.KEY_SERIALIZER_CLASS_CONFIG, StringSerializer.class);

I find it confusing when there is a random setting configured that is actually not used.

Is there a way to not set the "key.serializer" setting to signal that there is no key serialized?

Upvotes: 1

Views: 12281

Answers (3)

Sven Erik Knop
Sven Erik Knop

Reputation: 761

This is an old topic, but it still pops up on top of a Google search for similar topics, so just for completeness:

There is a VoidSerializer and a corresponding VoidSerde:

import org.apache.kafka.common.serialization.VoidSerializer;
import org.apache.kafka.common.serialization.Serdes.VoidSerde;

Use, for example, like this:

properties.put(ProducerConfig.KEY_SERIALIZER_CLASS_CONFIG, VoidSerializer.class);

Upvotes: 1

Jin Lee
Jin Lee

Reputation: 3522

Kafka broker stores byte arrays. So I think you must serialize.

Message of a Topic from a producer should be turned into bytes arrays(byte streams, streams of bytes) and stored in the partition of a broker. And then it gets de-serialized to go to Consumer.

And message is a key, value pair. so both key and value are required.

enter image description here

Upvotes: 3

Bartosz Wardziński
Bartosz Wardziński

Reputation: 6623

There is no option to indicate that the key will be null and key.serializer is not needed.

KafkaProducer doesn't know if the ProducerRecord that will be passed has key and/or value are set to null, so serializers have to be passed.

It is not possible to send a message without the key. Even if you use following constructor: ProducerRecord(String topic, V value). Behind the scene it sets a key (null): 

public ProducerRecord(String topic, V value) {
    this(topic, null, null, null /* key */, value, null);
}

Upvotes: 5

Related Questions