Reputation: 812
We are using Kafka Streams and Karpenter with normal Deployment in order to manage the pods for a service that we have.
After Karpenter decides to kill the pod, it brings a new Pod up, and we are seeing a delay when the new Pod takes over the processing of the partition.
That delay is around ~40 seconds.
I see the default value for normal Kafka Consumer session.timeout.ms
is 45 seconds - which makes us think that the Pod is not removed until the session is timed out by the broker.
Is the default value for kafka Streams session.timeout.ms
also 45 seconds?
If so that would explain the behavior that we have.
Apart from that, we are thinking of changing the internal.leave.group.on.close
property to true
- the default Kafka Streams one being false
.
Upvotes: 1
Views: 21