CODEWITHSUNDEEP
javaeclipsejarapache-kafkaspark-streaming
sirdan
sirdan

Reputation: 1028

NoClassDefFoundError: kafka/serializer/StringDecoder after exporting jar

I have exported two jars of two apps I have in the same Kafka/Spark Streaming project. The jar with the Kafka Producer works fine. The jar with the Spark Consumer returns this error:

NoClassDefFoundError: kafka/serializer/StringDecoder

They share the same dependencies folder, which I have obviously exported in order to make the jars work outside Eclipse.

In Eclipse they both work fine.

How can I fix this?

Upvotes: 1

Views: 1041

Answers (1)

Alexey Romanov
Alexey Romanov

Reputation: 170919

You need Kafka itself in the class path, not just spark-streaming-kafka. In Eclipse you do, because your build system retrieves all dependencies, and Eclipse integrates with it.

I looked into the jar of spark-streaming-kafka and it contains the .class file that is named in the error: org.apache.spark.streaming.kafka.KafkaUtils$.createStream(Ka‌​fkaUtils.scala:55). So I can't understand why he misses this class.

Presumably you mean that contains org.apache.spark.streaming.kafka.KafkaUtils$, but when JVM tries to load it, it discovers that it needs other classes, including kafka.serializer.StringDecoder, which is not there.

Upvotes: 2

Related Questions