CODEWITHSUNDEEP
javaapache-kafkaapache-kafka-streams
AnZzý
AnZzý

Reputation: 21

how to modify message of one kafka topic and send to another kafka topic using java?

I have created producer which produces MSG to one topic A and My need is I want to do change in that MSG and want to send it to another topic B, I'm trying to do this by Kafka streams,but not sure is it a right way or not. if it requires Kafka streams for that then please share what should be the code to write?

Upvotes: 1

Views: 6151

Answers (2)

Matthias J. Sax
Matthias J. Sax

Reputation: 62285

There are many options how you can to this.

Kafka's Streams API is one possibility with the advantage that you don't add a new dependency to your project. As simple stream.map(...).to(...) as program should do this. Check out the docs and examples for more details:

You could even just use plain KafkaConsumer and KafkaProducer to get this done (but using Kafka's Streams API seem to be more natural than this).

Of course, you can also use a stream processing framework like Apache Flink, Apache Storm, Apache Apex, Heron (or maybe Apache Spark) -- for sure, there are others too. The program would be similarly simple to write as sketched above. But you need to setup a processing cluster to deploy you job for those stream processing framework.

Upvotes: 9

Pritish Kamath
Pritish Kamath

Reputation: 159

You should look into using Apache storm, it will be very suitable for your needs.

I have used it and it does very well and gels like water in milk with kafka. It is also very easy to implement.

Best of luck

Upvotes: 1

Related Questions