CODEWITHSUNDEEP
javaspringspring-bootapache-kafkaapache-kafka-streams
Norbert Koch
Norbert Koch

Reputation: 541

Thread order for kafka consumer group threads differs on every start

I'm using kafka-streams 2.0.1 in Spring Boot 2.1.4 context. The CONSUMER-ID has the following pattern (autogenerated bei kafka-streams!?): 

<client.id>-StreamThread-<threadSequenceNumber>-<consumer|producer|restore-consumer> (source: StreamsConfig.java)

I figured out, that the value: threadSequenceNumber is at startup of my application not every time the same, because I have two StreamThread provided by Beans. I tried already to force the right order with annotating the bean with @DependsOn.

Everytime when the threadSequenceNumber changes the last offsets are lost. Anyone an idea how I can force the correct order?

Upvotes: 0

Views: 88

Answers (1)

Matthias J. Sax
Matthias J. Sax

Reputation: 62285

Your observation is correct. The thread sequence number is create by Kafka Streams. There is nothing Spring Boot can do about it.

Note, that this behavior will change in upcoming 2.3 release: https://issues.apache.org/jira/browse/KAFKA-8285

Everytime when the threadSequenceNumber changes the last offsets are lost.

Why this? Offsets are committed based on the application.id.

Upvotes: 1

Related Questions