Reputation: 1503
Is there a simple way to simulate a MapReduce job using the terminal? I wonder if there is an easy way to debug my code while writing it.
Upvotes: 2
Views: 664
Reputation: 1503
I'm used to Hadoop streaming with Python, but I believe this solution can be replicated for other languages as well. So, here it goes my solution:
cat input_folder/* | python map.py | sort | python reduce.py
It's also valid to mention that you can see the values your reducer receives as input, by simply running:
cat input_folder/* | python map.py | sort
Upvotes: 4