CODEWITHSUNDEEP
javahadoopmapreduce
Timofey Pivsaev
Timofey Pivsaev

Reputation: 510

Tuple as a key in Hadoop mapper

I need to use a "tuple" of 2 elements as a key in my mapreduce job.

I.e. I want my map task to output pair of (first_key,second_key), value

I tried using ArrayWritable, but after running my code I've got an error and it turned out that ArrayWritable isn't suitable for such task as it doesn't implement WritableComparable (whatever it means) and so, cannot be used as a key.

Strangely I cannot find any "right" solution for my problem that will work 100% with no strings attached. Any suggestions?

Upvotes: 3

Views: 2902

Answers (2)

Aleksei Shestakov
Aleksei Shestakov

Reputation: 2538

If you don't want to implement WritableComparable you could use new Text(key1.toString() + "\t" + key2.toString()) as key.

Upvotes: 1

Mouna
Mouna

Reputation: 3359

Any key in a MapReduce job must implement WritableComparable which not the case of ArrayWritable.

It must be Comparable too so that Hadoop can sort the (key,value) pairs. So in your case, a solution could be to create your own class that implements WritableComparable.

Upvotes: 2

Related Questions