CODEWITHSUNDEEP
hadoopmapreduce
divinedragon
divinedragon

Reputation: 5356

Use cases/example of Map only jobs

Are there any real life good use cases/examples for jobs which only involve only Map tasks and no reducers. A job which only triggers mappers and no reducers need to be set.

Upvotes: 2

Views: 1246

Answers (2)

Mark Giaconia
Mark Giaconia

Reputation: 3953

I have done many Map-Only Jobs.... here are a few examples.

  1. You have a classification model that you build every day, and you need to classify all your data with that classifier. There is no need for a reduce, you just load the classifier from the distributed cache (or from a remote resource like a DB) and inside the map() function of your mapper you do the classification and write the result somewhere.
  2. Performing data cleanup on something like an HBase table. Read in each row in your mapper, and if it matches some conditional statement then delete it. No need for reduce here.

Basically, if you don't need to combine or aggregate data, you just need to perform a repetitive serial process on each piece of data, you don't usually need a reducer. I would also say that if you don't need a reducer, then you might ask yourself if you might be better off with something like Apache Storm or another processing model with less overhead.

Upvotes: 5

Jean Logeart
Jean Logeart

Reputation: 53859

Sure!

Imagine that instead of the famous word-count problem, you simply replace each word by its length.

Doing so, you map each word to its length and you never reduce anything!

Hello map reduce would become 5 3 6

Upvotes: 1

Related Questions