CODEWITHSUNDEEP
javaspringspring-bootspring-cloud-stream
Devas
Devas

Reputation: 1694

How to remove the content type header from kafka messages produced by the spring cloud stream app starter sources

For twitter source it is creating a 30 bit header for each kafka messages if we use custom kafka consumer other than @StreamListener (Sink). @StreamListener sends the messages correctly to the annotated method (no header). The problem is for each source the header size varies. So how can I dynamically understand the header size of the messages since I'm using custom kafka consumer.? Or How can I remove the header for each message.?

UPDATE:-

Please find below the message received in kafka-console-consumer while using a TCP Source.

�
  contentType"application/octet-stream"hello world

I just need to get the message only. In this case, hello world

Upvotes: 4

Views: 5578

Answers (3)

zt1983811
zt1983811

Reputation: 1017

Could also change this setting to remove the annoying header

spring.cloud.stream.default.producer.useNativeEncoding=true

Upvotes: 0

Devas
Devas

Reputation: 1694

Setting spring.cloud.stream.default.producer.headerMode=raw in source will remove the header while reading the message in consumer.

Upvotes: 1

Oleg Zhurakousky
Oleg Zhurakousky

Reputation: 6126

What version of Kafka you are using? The reason why I am asking is that starting 0.11 Kafka supports native headers and you would not have this problem anymore.

Upvotes: 2

Related Questions