CODEWITHSUNDEEP
spring-bootapache-kafkaspring-cloud-streamspring-kafka
Omri Gelman
Omri Gelman

Reputation: 63

Spring cloud stream 2.0 and startOffset latest

While upgrading our app to spring boot 2.0.3 , we encoutered an issue with consuming messages and using startOffset: latest.
The scenario that we tested is as follows:

  1. deploy an app with consumer(topic = TEST_CHANNEL,group = kafka2_test) configured with startOffset: latest to pcf and increase the number of instances to 5.
  2. stop the app
  3. publish 40 messages to TEST_CHANNEL topic
  4. start the app

we can see that the app successfully assigns to the partitions ,however, some of the instances does not consume from the partitions that they are assigned to.
We also verified it by checking the lag for the group.
This scenario worked when using spring boot 1.5.10. It also worked when using spring boot 2.0.3 and not setting the startOffset: latest.

Note: this is not a new group , so offset suppose to be present for the consumer to use, and resetOffsets should not have any effect.

Is this an issue in spring boot 2.0.3?

Upvotes: 3

Views: 1190

Answers (1)

Gary Russell
Gary Russell

Reputation: 174779

not a new group ... and resetOffsets should not have any effect.

resetOffsets is specifically designed to reset the offset for an existing group to the startOffset value.

It was broken in the 1.3.x version of the binder (the version used by Boot 1.5.x) and had no effect.

Upvotes: 2

Related Questions