CODEWITHSUNDEEP
apache-kafkaspring-integrationspring-kafka
Okay Atalay
Okay Atalay

Reputation: 111

Spring Integration Kafka Partition change detection

I have a problem to detect kafka partition changes in runtime. I configured kafka using spring integration and i could not figure out how to detect number of partition changes during the application is running.

The main problem is that the kafka topic has 10 partitions and my kafka config is below(it has 10 threads. 1 to 1 relationship between partition and thread). when i increase the kafka topic partition number(assuming 20 partitions), application could not read messages which come into newly created partitions without restarting.

is there any way to configure spring integration to be aware of this kind of changes?

Thanks in advance.

 IntegrationFlow flow = IntegrationFlows.from(Kafka.messageDrivenChannelAdapter(kafkaConsumerFactory, topic)
                      .configureListenerContainer(c-> c.concurrency(10))))
            .transform(transformer)
            .get();
....

Upvotes: 0

Views: 123

Answers (1)

OneCricketeer
OneCricketeer

Reputation: 191738

This happens automatically, every metadata.max.age.ms Consumer property (default 5 minutes)

Upvotes: 1

Related Questions