CODEWITHSUNDEEP
javaapache-kafkaapache-kafka-streams
R. B
R. B

Reputation: 175

Receiving exception when trying to run simple Kafka Stream App

I have been trying to run a simple wordcount application using Kafka however whenever I run it, I get the following error: 

Exception in thread "main" java.lang.NoClassDefFoundError: org/apache/kafka/common/utils/LogContext
    at org.apache.kafka.streams.KafkaStreams.<init>(KafkaStreams.java:630)
    at org.apache.kafka.streams.KafkaStreams.<init>(KafkaStreams.java:610)
    at org.apache.kafka.streams.KafkaStreams.<init>(KafkaStreams.java:557)
    at StreamsApp.main(StreamsApp.java:49)
    at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
    at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.base/java.lang.reflect.Method.invoke(Method.java:566)
    at com.intellij.rt.execution.application.AppMainV2.main(AppMainV2.java:131)
Caused by: java.lang.ClassNotFoundException: org.apache.kafka.common.utils.LogContext
    at java.base/jdk.internal.loader.BuiltinClassLoader.loadClass(BuiltinClassLoader.java:583)
    at java.base/jdk.internal.loader.ClassLoaders$AppClassLoader.loadClass(ClassLoaders.java:178)
    at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:521)

I am not sure why I keep getting this error... The code for the main method is listed below. (Line 49) KafkaStreams streams = new KafkaStreams(topology, props);

public static void main(final String[] args) throws Exception {

    Properties props = new Properties();
    props.put(StreamsConfig.APPLICATION_ID_CONFIG, "wordcount-application");
    props.put(StreamsConfig.BOOTSTRAP_SERVERS_CONFIG, "localhost:9092");
    props.put(StreamsConfig.DEFAULT_KEY_SERDE_CLASS_CONFIG, Serdes.String().getClass().getName());
    props.put(StreamsConfig.DEFAULT_VALUE_SERDE_CLASS_CONFIG, Serdes.String().getClass().getName());
    props.put(ConsumerConfig.AUTO_OFFSET_RESET_CONFIG, "earliest");
    // props.put(StreamsConfig.STATE_DIR_CONFIG, TestUtils.tempDirectory().getAbsolutePath());

    StreamsBuilder builder = new StreamsBuilder();
    KStream<String, String> textLines = builder.stream("inputTopic");
    final Pattern pattern = Pattern.compile("\\W+", Pattern.UNICODE_CHARACTER_CLASS);

    KTable<String, Long> wordCounts = textLines
            .flatMapValues(value -> Arrays.asList(pattern.split(value.toLowerCase())))
            .groupBy((key, word) -> word)
            .count();

    wordCounts
            .foreach((w, c) -> System.out.println("word: " + w + " -> " + c));


    String outputTopic = "outputTopic";
    Serde<String> stringSerde = Serdes.String();
    Serde<Long> longSerde = Serdes.Long();
    wordCounts.to(stringSerde, longSerde, outputTopic);

    Topology topology = builder.build();

    KafkaStreams streams = new KafkaStreams(topology, props);
    streams.start();

    Thread.sleep(30000);
    streams.close();

}

}

Upvotes: 1

Views: 7796

Answers (3)

MaduKan
MaduKan

Reputation: 660

Its strange why the kafka-stream required separately. I thought it was included in the ksqldb. However following is what fixed the issue for me. Please shout if this isn't correct!

    <dependency>
        <groupId>io.confluent.ksql</groupId>
        <artifactId>ksqldb-api-client</artifactId>
        <version>0.27.2</version>
    </dependency>
    <dependency>
        <groupId>org.apache.kafka</groupId>
        <artifactId>kafka-streams</artifactId>
        <version>3.2.1</version>
    </dependency>

Upvotes: 1

Sashikant Prasad
Sashikant Prasad

Reputation: 61

Required jars should be in class path. Hope below can help you.

C:\PROJECT\ENVIRONMENT\KAFKA_2_11\bin\windows\kafka-run-class.bat -cp  C:\PROJECT\KafkaDemos\target\classes;C:\PROJECT\ENVIRONMENT\KAFKA_2_11\libs\kafka_2.11-2.4.0.jar;C:\PROJECT\ENVIRONMENT\KAFKA_2_11\libs\kafka-clients-2.4.0.jar;C:\PROJECT\E
NVIRONMENT\KAFKA_2_11\libs\kafka-streams-2.4.0.jar;C:\PROJECT\ENVIRONMENT\KAFKA_2_11\libs\kafka-clients-2.4.0.jar;C:\PROJECT\ENVIRONMENT\KAFKA_2_11\libs\slf4j-api-1.7.28.jar;C:\PROJECT\ENVIRONMENT\ExtraJars\apache-logging-log4j.jar;C:\PROJECT\ENVIRONMENT\ExtraJars\
rocksdbjni-6.5.3.jar; demo.wordcount.WordCount

Upvotes: 0

Vasyl Sarzhynskyi
Vasyl Sarzhynskyi

Reputation: 3955

you have conflicts between dependencies org.apache.kafka:kafka-streams and org.apache.kafka:kafka-clients. according to your exception, you use kafka-clients version less than 1.0.0, but kafka-streams version is equal or higher than 1.0.0.

make sure that kafka-clients version is at least 1.0.0 (so you need to upgrade your kafka-clients version), otherwise you need to downgrade kafka-streams version.

Upvotes: 8

Related Questions