CODEWITHSUNDEEP
scalaapache-sparkhdfs
sirine
sirine

Reputation: 527

How to save mllib.recommendation.MatrixFactorizationModel into HDFS?

I want to save my model of training in HDFS, I tried by:

model.saveAsTextFile("hdfs://sandbox.hortonworks.com:8020/tmp/Project/Model")

and

model.saveAsTextFile("hdfs:///sandbox.hortonworks.com:8020/tmp/Project/Model")

But it dispaly an error is the following:

value saveAsTextFile is not a member of org.apache.spark.mllib.recommendation.MatrixFactorizationModel

Can I use saveAsnewAPIHadoopFile("hdfs://....") ?

Can you tell how to save the model into HDFS.

Upvotes: 2

Views: 261

Answers (1)

Jacek Laskowski
Jacek Laskowski

Reputation: 74679

In the scaladoc of MatrixFactorizationModel I could only find the method:

save(sc: SparkContext, path: String): Unit

Use save instead.

save(sc: SparkContext, path: String): Unit

Save this model to the given path.

This saves:

  • human-readable (JSON) model metadata to path/metadata/
  • Parquet formatted data to path/data/

You seem to have used ALS. If so, please consider using org.apache.spark.ml.recommendation package (not org.apache.spark.mllib.recommendation).

Upvotes: 1

Related Questions