CODEWITHSUNDEEP
javaapache-kafkaapache-kafka-streams
ethrbunny
ethrbunny

Reputation: 10479

Kafka KStream - significant startup delay

Im running into an issue with my KStreams based application: it will run once and when I stop/restart it gets 'stuck' and won't progress anymore until I delete the various topics it has created. This doesn't happen every time but more often than not.

Typically this happens when I copy a new(er) version to the work VM (in the same subnet as the kafka cluster for speed reasons).

When it's wedged I'll see;

  1. "Connect": org.apache.zookeeper.ZooKeeper - Initiating client connection
  2. "Client": [StreamThread-1] INFO o.a.k.s.p.internals.StreamTask - Creating restoration consumer client
  3. "Ping" : I'll see these and the app won't shut down normally. It must be kill'd.

In any of these cases the message will typically repeat indefinitely (well - at least all the way through a lunch + meeting. IE Too long).

The app is shutting down 'cleanly' before this happens.

What am I doing wrong?

Edit:

This most recent time - after 20 minutes I got a stream of errors:

org.apache.kafka.common.errors.TimeoutException: Batch containing 101 record(s) expired due to timeout while requesting metadata from brokers

followed by:

org.apache.kafka.clients.consumer.CommitFailedException: Commit cannot be completed since the group has already rebalanced and assigned the partitions to another member

--> which is a good trick since there is no other member.

Upvotes: 0

Views: 1365

Answers (1)

Guozhang Wang
Guozhang Wang

Reputation: 489

If you are running with Kafka 0.10.0.x then you may be hitting a known issue:

https://cwiki.apache.org/confluence/display/KAFKA/KIP-62%3A+Allow+consumer+to+send+heartbeats+from+a+background+thread

This has been resolved in the upcomming 0.10.1.0 release of Kafka, and I would recommend you trying out the new version to see if this issue goes away.

Upvotes: 2

Related Questions