Reputation: 16489
Pretty new to HDFS:
Correct me if I am wrong but to my knowledge:
Mapper<Object, Text, Text, Text>
The last two Text
are the return types of the Mapper. Say I wanted to return Text, <Text, IntWriteable>
. How could I achieve this? Also, where in the documentation should I look?
For example, after the mapper receives the Object and Text, it does some logic with the lines of data in the input file, I want it to return something like
context.write(Text,[Text, IntWriteable])
Upvotes: 0
Views: 1434
Reputation: 4999
You may need to define your own data type. If comparison operation needed, implement WritableComparable interface, otherwise just implement Writable interface is ok.
Here is a sample using WritableComparable
interface:
public class MyDataType implements WritableComparable<MyDataType> {
private Text name;
private IntWritable age;
@Override
public void readFields(DataInput in) throws IOException {
name.readFields(in);
age.readFields(in);
}
@Override
public void write(DataOutput out) throws IOException {
name.write(out);
age.write(out);
}
@Override
public int compareTo(MyDataType o) {
int nameCompare = name.compareTo(o.name);
if(nameCompare != 0) {
return nameCompare;
} else {
return age.compareTo(o.age);
}
}
}
Upvotes: 1
Reputation: 1082
you have to implement the custom writable in hadoop.I am pointing some sites,might be helpful
https://halalhassan.wordpress.com/2013/12/15/custom-writables-in-hadoop-the-right-way/ http://www.hadoopmaterial.com/2013/10/custom-hadoop-writable-data-type.html
Upvotes: 1