CODEWITHSUNDEEP
google-app-enginemapreducegoogle-app-engine-python
Erfa
Erfa

Reputation: 713

How do you send parameters to the mapper function in mapreduce on Google App Engine?

I'm trying to figure out how to use the mapreduce library with the DatastoreInputReader on App Engine. I have had a lot of help from this excellent tutorial: https://sookocheff.com/post/appengine/mapreduce/programmatic-mapreduce/

But there's one thing I can't find any examples of - how do you send custom parameters to the mapper function? It seems like it only receives the entity. What if I also want to access some other data relevant to the job? For example, if I have a game and want to set everyone's score to a specific value, can I send that in somewhere? Or do I have to create a separate mapper function for every value I want to use, like this?

def mapper_10(user):
    user.score = 10
    user.put()

def mapper_50(user):
    user.score = 50
    user.put()

This looks very silly, obviously. Is there no way to do something like this?

def mapper(user, new_score):
    user.score = new_score
    user.put()

Or do I have to create my own input reader for that?

Upvotes: 0

Views: 80

Answers (1)

Erfa
Erfa

Reputation: 713

Turns out it was possible, but a bit more tricky than I thought. While you can't receive parameters to the method, you can send in parameters to the pipeline and then fetch them through the mapreduce context.

from mapreduce import context

def mapper(user):
    user.score = ctx.mapreduce_spec.mapper.params['new_score']
    user.put()

Problem solved!

Upvotes: 1

Related Questions