CODEWITHSUNDEEP
apache-kafkaapache-kafka-streamskafka-producer-api
Nandish Kotadia
Nandish Kotadia

Reputation: 461

Kafka Stream - Uncaught error in kafka producer I/O thread: java.util.ConcurrentModificationException: null

I am running a Kafka Stream application and recently I have started experiencing below exception and kafka stream process goes to pending shutdown state. This shows exception in kafka producer internal API code. Can it be because of the heavy load on the kafka brokers?

2019-08-12 10:54:30 - [ERROR] [kafka-producer-network-thread | c8-max-view-live-1-StreamThread-1-producer] [org.apache.kafka.clients.producer.internals.Sender.run:235] : [Producer clientId=c8-max-view-live-1-StreamThread-1-producer] Uncaught error in kafka producer I/O thread:
java.util.ConcurrentModificationException: null
    at java.util.HashMap$HashIterator.nextNode(HashMap.java:1429)
    at java.util.HashMap$EntryIterator.next(HashMap.java:1463)
    at java.util.HashMap$EntryIterator.next(HashMap.java:1461)
    at org.apache.kafka.clients.producer.internals.Sender.getExpiredInflightBatches(Sender.java:177)
    at org.apache.kafka.clients.producer.internals.Sender.sendProducerData(Sender.java:353)
    at org.apache.kafka.clients.producer.internals.Sender.run(Sender.java:308)
    at org.apache.kafka.clients.producer.internals.Sender.run(Sender.java:233)
    at java.lang.Thread.run(Thread.java:745)
2019-08-12 10:54:30 - [ERROR] [kafka-producer-network-thread | c8-max-view-live-1-StreamThread-1-producer] [org.apache.kafka.clients.producer.internals.Sender.run:235] : [Producer clientId=c8-max-view-live-1-StreamThread-1-producer] Uncaught error in kafka producer I/O thread:
java.util.ConcurrentModificationException: null
    at java.util.HashMap$HashIterator.nextNode(HashMap.java:1429)
    at java.util.HashMap$EntryIterator.next(HashMap.java:1463)
    at java.util.HashMap$EntryIterator.next(HashMap.java:1461)
    at org.apache.kafka.clients.producer.internals.Sender.getExpiredInflightBatches(Sender.java:177)
    at org.apache.kafka.clients.producer.internals.Sender.sendProducerData(Sender.java:353)
    at org.apache.kafka.clients.producer.internals.Sender.run(Sender.java:308)
    at org.apache.kafka.clients.producer.internals.Sender.run(Sender.java:233)
    at java.lang.Thread.run(Thread.java:745)

After this kafka streams process hangs:

2019-08-12 10:54:31 - [INFO] [c8-max-view-live-1-StreamThread-1] [org.apache.kafka.streams.KafkaStreams.setState:257] : stream-client [c8-max-view-live-1] State transition from ERROR to PENDING_SHUTDOWN
2019-08-12 10:54:31 - [INFO] [kafka-streams-close-thread] [org.apache.kafka.streams.processor.internals.StreamThread.shutdown:1164] : stream-thread [c8-max-view-live-1-StreamThread-1] Informed to shut down

Upvotes: 1

Views: 2737

Answers (1)

Shailendra
Shailendra

Reputation: 9102

One bug around this issue of mutating underlying Collection while being iterated was identified and fixed. Please check here.

Upvotes: 1

Related Questions