CODEWITHSUNDEEP
javaapache-kafka
TechStack
TechStack

Reputation: 1

Saving data to data base while consuming through multiple consumers from a kafka

I am new to kafka the problem i am facing is that I have 2 consumer that are consuming the data in batches from the kafka topic and want to save all the data fetched by each consumer to db. How to achieve that?
In the properties file listner.type=batch and max-poll-records=500 is set. I have created a Save message method that is defined in the service class and i am calling the same method from both the consumer. While saving the records it is taking too much time.

Upvotes: 0

Views: 462

Answers (1)

OneCricketeer
OneCricketeer

Reputation: 191864

If your consumer is "taking too long" and "doing too much", then rebalancing the consumer group and making no progress, then you simply need smaller batches rather than 500.

Otherwise, you need to improve the network connection between your app and the database, which is a hardware problem.

Upvotes: 1

Related Questions