CODEWITHSUNDEEP
hadoopapache-sparkmahout
HHH
HHH

Reputation: 6465

How to run Mahout on Spark

I recently noticed that there are some discussions on running Mahout algorithms on Spark instead of MapReduce. However I couldn't find any documentation for that. Could someone tell me whether it is possible to run Mahout algorithms on Spark? If so, is there any restriction on the algorithms that we can run?

Upvotes: 0

Views: 998

Answers (1)

Shyamendra Solanki
Shyamendra Solanki

Reputation: 8851

Yes, mahout now runs on Spark (i.e. the new version v0.10.0).

The algorithms available on different engines are documented here.

These are available on Spark:

Mahout Distributed BLAS. Distributed Row Matrix API with R and Matlab like operators. Distributed ALS, SPCA, SSVD, thin-QR. Similarity Analysis. User/Item-Based Collaborative Filtering. Naive Bayes Classification. Stochastic SVD. PCA (via Stochastic SVD). QR Decomposition. RowSimilarityJob.

Here is one of the algorithms that can be used:

mahout spark-itemsimilarity           \
       -i /mnt/similarity/input       \
       -o /mnt/similarity/output      \
       --master yarn-client           \
       -D:spark.executor.instances=10 \
       -D:spark.executor.cores=8

Upvotes: 2

Related Questions