Hamide Ahadi
Hamide Ahadi

Reputation: 56

KryoException: Buffer underflow error in Apache Storm and Storm-Crawler

I have been encountering a recurring issue during the deployment of a new version of my topology in Storm-Crawler, and I am seeking assistance in understanding and resolving the problem.

Error: Upon deployment, I consistently encounter the following error:

Buffer underflow.
Serialization trace:
value (org.apache.storm.metric.api.IMetricsConsumer$DataPoint)

Stack Trace:
com.esotericsoftware.kryo.io.Input in require at line 199
com.esotericsoftware.kryo.io.Input in readVarInt at line 373
com.esotericsoftware.kryo.util.DefaultClassResolver in readClass at line 127
com.esotericsoftware.kryo.Kryo in readClass at line 670
com.esotericsoftware.kryo.Kryo in readClassAndObject at line 781
com.esotericsoftware.kryo.serializers.MapSerializer in read at line 153
com.esotericsoftware.kryo.serializers.MapSerializer in read at line 39
....
com.esotericsoftware.kryo.Kryo in readObject at line 689
org.apache.storm.serialization.KryoValuesDeserializer in deserializeFrom at line 31
org.apache.storm.serialization.KryoTupleDeserializer in deserialize at line 45

This error occurs between two nodes of the storm cluster, for example nodes 3 and 8. It seems that node 3 sends a tuple to node 8 and node 8 experiences Buffer underflow and starts to reset the worker.

Despite investing several hours in troubleshooting and conducting various tests, I have been unable to determine the root cause of this issue. It is worth mentioning that I have thoroughly tested the version on my local environment and confirmed its functionality without any problems.

I kindly request your insights, suggestions, or any guidance on how to effectively diagnose and resolve this persistent deployment error.

Thank you in advance for your assistance and support.

Upvotes: 3

Views: 111

Answers (0)

Related Questions