CODEWITHSUNDEEP
apache-sparknaivebayes
Fleur
Fleur

Reputation: 686

Spark Naive Bayes Model - No such method error

I'm new to Spark and am trying to run the Naive Bayes Java example in Eclipse. It gives a No such method error when trying to save the model. 

SparkConf sparkConf = new SparkConf().setAppName("JavaNaiveBayesExample").setMaster("local");
JavaSparkContext jsc = new JavaSparkContext(sparkConf);

String path = "SPARK_HOME\\data\\mllib\\sample_libsvm_data.txt";
JavaRDD<LabeledPoint> inputData = MLUtils.loadLibSVMFile(jsc.sc(), path).toJavaRDD();
JavaRDD<LabeledPoint>[] tmp = inputData.randomSplit(new double[]{0.6, 0.4});
JavaRDD<LabeledPoint> training = tmp[0]; // training set
JavaRDD<LabeledPoint> test = tmp[1]; // test set
final NaiveBayesModel model = NaiveBayes.train(training.rdd(), 1.0);
JavaPairRDD<Double, Double> predictionAndLabel =
  test.mapToPair(new PairFunction<LabeledPoint, Double, Double>() {
      public Tuple2<Double, Double> call(LabeledPoint p) {
      return new Tuple2(model.predict(p.features()), p.label());
    }
  });
double accuracy = predictionAndLabel.filter(new Function<Tuple2<Double, Double>, Boolean>() {
  public Boolean call(Tuple2<Double, Double> pl) {
    return pl._1().equals(pl._2());
  }
}).count() / (double) test.count();

// Save and load model    
model.save(jsc.sc(), "PATH_TO_FOLDER\\myNaiveBayesModel");
NaiveBayesModel sameModel = NaiveBayesModel.load(jsc.sc(), "PATH_TO_FOLDER\\myNaiveBayesModel");

jsc.close();

The error occurs in model.save() line.

The stack trace is as below:

Exception in thread "main" java.lang.NoSuchMethodError: scala.reflect.api.JavaUniverse.runtimeMirror(Ljava/lang/ClassLoader;)Lscala/reflect/api/JavaMirrors$JavaMirror;
at org.apache.spark.mllib.classification.NaiveBayesModel$SaveLoadV2_0$.save(NaiveBayes.scala:205)
at org.apache.spark.mllib.classification.NaiveBayesModel.save(NaiveBayes.scala:170)
at AppMain.main(AppMain.java:42)

How can I resolve this? Any help is appreciated.

Running Spark 2.0.1 and Scala 2.11.8.

Dependencies added for spark-core_2.11 and spark-mllib_2.10.

Upvotes: 0

Views: 344

Answers (1)

Rakesh Rakshit
Rakesh Rakshit

Reputation: 592

Dependencies added for spark-core_2.11 and spark-mllib_2.10.

Please use spark mllib compiled with Scala 2.11 to resolve the issue.

Upvotes: 2

Related Questions