CODEWITHSUNDEEP
mysqlmahoutevaluationrecommendation-enginemahout-recommender
Nick
Nick

Reputation: 19

Apache mahout evaluation extremely slow - with modified NearestNUserNeighborhoud

I have modified the NearestNUserNeighborhoud to add some extra value to users that belong to a trust network. The trust network is in a database (MySQL).

When I recommend items for a certain users it works well and provides the recommendations.

However when I evaluate RMSE is extremely slow when it make the comparisons of users that are in the database / trust network of the user. (It runs thousand of select statements)

The first data model is on a csv file and it is on the form userid itemid rating.Whereas in the database is in the form truster trustee.

What would you suggest to evaluate RMSE in a reasonable amount of time?

Upvotes: 0

Views: 167

Answers (1)

Dragan Milcevski
Dragan Milcevski

Reputation: 776

The mahout evaluation algorithm can take advantage of parallel processing. You can run it on bigger machine with more cores. However, it seems that you problem is the database access. Make sure you have one prepare statement and just call the database. (Do not do prepare statement all the time).

Better solution will be to load your trust users in memory in Map (Use Trove library, it consumes less memory), and do your check in memory, instead from the database.

Upvotes: 1

Related Questions