CODEWITHSUNDEEP
apache-kafkakafka-producer-api
tosi
tosi

Reputation: 1925

Is there a way to add headers in kafka-console-producer.sh

I'd like to use the kafka-console-producer.sh to fire a few JSON messages with Kafka headers.

Is this possible?

docker exec -it kafka_1 /opt/kafka_2.12-2.3.0/bin/kafka-console-producer.sh --broker-list localhost:9093 --topic my-topic --producer.config /opt/kafka_2.12-2.3.0/config/my-custom.properties

Upvotes: 17

Views: 19869

Answers (3)

dswiecki
dswiecki

Reputation: 388

Starting from kafka 3.1.0, there is an option to turn on headers parsing parse.headers=true and then you just place them before your record value. Info from docs:

| parse.headers=true:
|  "h1:v1,h2:v2...\tvalue"

So your command will look like this:

kafka-console-producer.sh --bootstrap-server localhost:9092 --topic topic_name --property parse.headers=true

and then you should pass:

header_name:header_value\trecord_value

Upvotes: 13

Sambhav Khare
Sambhav Khare

Reputation: 71

Consumer producer command:

docker exec -it container_id kafka-console-producer.sh --topic my-topic --bootstrap-server localhost:9092 --property parse.headers=true --property headers.key.separator=: --property headers.delimiter=\t

Payload format:

header_name:header_value t payload_value

Console consumer command:

docker exec -it container_id kafka-console-consumer.sh --topic my-topic --bootstrap-server localhost:9092 --property print.headers=true

Upvotes: 1

Robin Moffatt
Robin Moffatt

Reputation: 32100

No, but you can with kafkacat's -H argument:

Produce:

echo '{"col_foo":1}'|kafkacat -b localhost:9092 -t test -P -H foo=bar

Consume:

kafkacat -b localhost:9092 -t test -C -f '-----\nTopic %t[%p]\nOffset: %o\nHeaders: %h\nKey: %k\nPayload (%S bytes): %s\n'
-----
Topic test[0]
Offset: 0
Headers: foo=bar
Key:
Payload (9 bytes): col_foo:1
% Reached end of topic test [0] at offset 1

Upvotes: 16

Related Questions