CODEWITHSUNDEEP
javaapache-spark
Fleur
Fleur

Reputation: 686

How to to retrieve the saved ALS model using Spark - Java

I am training an ALS model using a dataset. I am trying to save this model and later retrieve it and use it to get predictions. I can save the model but when trying to retrieve it get an error as IllegalAccessError: tried to access method com.google.common.base.Stopwatch.<init>()V from class org.apache.hadoop.mapred.FileInputFormat.

My code is as below:

    MatrixFactorizationModel model;
    MatrixFactorizationModel sameModel;

    int rank = 10;
    int numIterations = 10;
    model = ALS.train( JavaRDD.toRDD( ratings ), rank, numIterations, 0.01 );
    model.save( sc.sc(), "src/main/resources/UserBasedModel" );
    sameModel = MatrixFactorizationModel.load( sc.sc(), "src/main/resources/UserBasedModel" );

The stack trace is as below:

stack trace

How can I solve this? Any help is appreciated.

Upvotes: 1

Views: 433

Answers (1)

Fleur
Fleur

Reputation: 686

Setting Hadoop version to 2.7.2 solved the problem.

As explained in the answer for the SO question in here the error occurs due to the mismatch in Hadoop and Guava versions. For Guava versions above 17.0, Hadoop 2.7.2 has to be used.

Upvotes: 1

Related Questions