CODEWITHSUNDEEP
scalaapache-kafka
twes
twes

Reputation: 307

Trying to build and run Apache Kafka 0.8 against Scala 2.9.2 without success

As stated in the topic description, I'm trying to get Kafka 0.8 running with Scala 2.9.2.

I was able to get a working version using the quick start for 0.8 (https://cwiki.apache.org/KAFKA/kafka-08-quick-start.html), but it is compiled against Scala 2.8.0 by default.

I tried to modify the step

./sbt package

to 

./sbt "++2.9.2 package"

it compiles without errors but during start it complains that it cannot find the main class.

/tmp/kafka-8-test/kafka[0.8]$ bin/kafka-server-start.sh onfig/server1.properties
Error: Could not find or load main class kafka.Kafka

Any help will be highly appreciated.

Upvotes: 4

Views: 6459

Answers (3)

prenomenon
prenomenon

Reputation: 130

The problem is that the bin/kafka-server-start.sh script uses bin/kafka-run-class.sh to execute the generated jar file.

This script has hard-coded versions, so you need to customize it like this:

...
library=$(echo "$ivyPath/org.scala-lang/scala-library/jars/scala-library-2.9.2.jar")
CLASSPATH=$CLASSPATH:$library

compiler=~$(echo "$ivyPath/org.scala-lang/scala-compiler/jars/scala-compiler-2.9.2.jar")
CLASSPATH=$CLASSPATH:$compiler

log4j=$(echo "$ivyPath/log4j/log4j/jars/log4j-1.2.15.jar")
CLASSPATH=$CLASSPATH:$log4j

slf=$(echo "$ivyPath/org.slf4j/slf4j-api/jars/slf4j-api-1.6.4.jar")
CLASSPATH=$CLASSPATH:$slf

zookeeper=$(echo "$ivyPath/org.apache.zookeeper/zookeeper/jars/zookeeper-3.3.4.jar")
CLASSPATH=$CLASSPATH:$zookeeper

jopt=$(echo "$ivyPath/net.sf.jopt-simple/jopt-simple/jars/jopt-simple-3.2.jar")
CLASSPATH=$CLASSPATH:$jopt

for file in $base_dir/core/target/scala-2.9.2/*.jar;
do
  CLASSPATH=$CLASSPATH:$file
done

for file in $base_dir/core/lib/*.jar;
do
  CLASSPATH=$CLASSPATH:$file
done

for file in $base_dir/perf/target/scala-2.9.2/kafka*.jar;
do
  CLASSPATH=$CLASSPATH:$file
done
...

Upvotes: 4

peter_v
peter_v

Reputation: 392

With the newest version of Kafka 0.8.1.1 and gradlew the "SCALA_VERSION" is a variable in the script.

SCALA_VERSION=2.10.4

However, somewhere something goes wrong after:

``` ./gradlew -PscalaVersion=2.10.4 jar

```

where one of the files does not have the 2.10.4 part, but only 2.10 :

peter_v@trusty64:~/data/kafka/kafka-0.8.1.1-src$ find . -name '*.jar' ./perf/build/libs/kafka-perf_2.10-0.8.1.1.jar ./clients/build/libs/kafka-clients-0.8.1.1.jar ./system_test/migration_tool_testsuite/0.7/lib/kafka-perf-0.7.0.jar ./system_test/migration_tool_testsuite/0.7/lib/kafka-0.7.0.jar ./system_test/migration_tool_testsuite/0.7/lib/zkclient-0.1.jar ./examples/build/libs/kafka-examples-0.8.1.1.jar ./core/build/libs/kafka_2.10-0.8.1.1.jar ############ 2.10 instead of 2.10.4 ? ./core/build/dependant-libs-2.10.4/snappy-java-1.0.5.jar ./core/build/dependant-libs-2.10.4/metrics-core-2.2.0.jar ./core/build/dependant-libs-2.10.4/zkclient-0.3.jar ./core/build/dependant-libs-2.10.4/log4j-1.2.15.jar ./core/build/dependant-libs-2.10.4/slf4j-api-1.7.2.jar ./core/build/dependant-libs-2.10.4/zookeeper-3.3.4.jar ./core/build/dependant-libs-2.10.4/jopt-simple-3.2.jar ./core/build/dependant-libs-2.10.4/scala-library-2.10.4.jar ./target/scala-2.10/kafka-0-8-1-1-src_2.10-0.1-SNAPSHOT.jar ./lib/apache-rat-0.8.jar ./contrib/hadoop-consumer/lib/piggybank.jar ./contrib/hadoop-consumer/build/libs/kafka-hadoop-consumer-0.8.1.1.jar ./contrib/hadoop-producer/lib/piggybank.jar ./contrib/hadoop-producer/build/libs/kafka-hadoop-producer-0.8.1.1.jar ./contrib/build/libs/contrib-0.8.1.1.jar ./gradle/wrapper/gradle-wrapper.jar

With a copy to the 2.10.4 name as a workaround, Kafka started correctly.

```

cp core/build/libs/kafka_2.10-0.8.1.1.jar core/build/libs/kafka_2.10.4-0.8.1.1.jar ```

Upvotes: 0

user1779032
user1779032

Reputation: 496

kafka-run-class.sh is hard-coded to Scala 2.8.0. You can change 2.8.0 to 2.9.2 as suggested by prenomenon.

This works for me :

  • Linux|Unix

    sed -i "s/2.8.0/2.9.2/g" bin/kafka-run-class.sh

  • MacOS

    sed -i.bak "s/2.8.0/2.9.2/g" bin/kafka-run-class.sh

Upvotes: 9

Related Questions