CODEWITHSUNDEEP
apache-spark-mllibrecommendation-enginebigdata
Rohan
Rohan

Reputation: 11

How can I recommend content(say movies) based on multiple parameters

I am new to Spark ML. I want to recommend movies to users using Apache Spark ML. I learned here that we can recommend movies based on rating by a user.

My question is can we include other features for recommendation, say, his age, country, genre of movie, likes, etc. For example, we have users 'U1', 'U2', 'U3' and 'U4' all of them watching Movie 'M'. U1 of age 22 living in America also watched movie M1, M2 and M3 and U2 of age 50 living in Australia watched movie M1, M4 and M5. Now, I would like to recommend U3 of age 24 living in America movies M1, M2 and M3. Also Recommend movies M1,M4 and M5 to U4 of age 21 living in Australia.

Basically, I want to provide some weights to age and country. How can we achieve this with Spark ml(say using ALS)?

Upvotes: 1

Views: 1213

Answers (2)

Bartłomiej Twardowski
Bartłomiej Twardowski

Reputation: 640

This problem of introducing a context information can be addressed in a few different ways:

  • if you have to stick to Spark and 2D (user x item) recommender: then go for a contextual post/pre filtering. As you have the MF with ALS solver already there, you can use it as model base recommender in two ways for what you are trying to achieve: (1/pre-filtering) group your data by you context (e.g. age ranges, country) before creating 2D MF-ALS recommender for each of group or (2/post-filtering) create on single 2D user-item recommender an then do the post-filtering based on your contextual information after getting recommendations from your model. In the second option, you may try not only the boolean filtering but also boosting score value based on the contextual match, e.g. boost based on the avg(age of the watchers).
  • use contextual information directly into a model based recommender. This one cannot be directly applied with core/MLlib Spark matrix factorization method. This one is prepared for 2D, user-item actions. But to overcome this you can use different model-based method. I would advice going for Factorization Machines as it is a successful model for context-aware recommendations and it's very intuitive, especially in data preparation. You can see introduction from my slides here: Intro to FM slides. There is a package for Apache Spark - Spark libFM Package, but cannot say anything about it - is it really working or still maintained (?). But for sure can advice main libfm.org, python - fastFM and very similar+fast implementation lightfm from Lyst, which can be useful for your problem.
  • do kNN recommender where you introduce your own similarity measures while comparing user2user. It's heuristic base, where you can boost similarity score where the contextual information about both user match - e.g. using sth like: abs(u1.age - u2.age)/(max(age) - min(age), country, city or region distance between u1 and u2.

I would also advice to read the chapter from Recommender Systems Handbook[6] about introducing contextual information to recommender systems. And of course the whole book!

[6]: Kantor, Paul B. Recommender systems handbook. Eds. Francesco Ricci, Lior Rokach, and Bracha Shapira. Berlin, Germany:: Springer, 2015.

Upvotes: 5

Chandan Purbia
Chandan Purbia

Reputation: 285

I made a project on movie recommendation system using Apache Spark MLlib. I had help from codementor. You can find the tutorial here.

Upvotes: 0

Related Questions