CODEWITHSUNDEEP
springsolrapache-kafkadistributed
oortcloud_domicile
oortcloud_domicile

Reputation: 850

How to make Spring kafka client distributed

I have messages coming in from Kafka. So I am planning to write a listener and "onMessage". I want to process it and push it in to solr.

So my question is more architectural, like I have worked on web apps all my career, so in big data how to deploy the spring kafka listener, so I can process thousands of messages a second.

  1. How do I make my spring code use multiple nodes to distribute the load?
  2. I am planning to write a SpringBoot application to run in a tomcat container.

Upvotes: 1

Views: 144

Answers (1)

Natalia
Natalia

Reputation: 4532

If you use the same group id for all instances, different partitions will be assigned to different consumers (instances of your application).

enter image description here

So, be sure that you specified enough partitions in the topic you are going to consume.

Upvotes: 1

Related Questions