CODEWITHSUNDEEP
hadoopmapreduce
user3168080
user3168080

Reputation: 11

sorting numbers using hadoop mapreduce

The data looks like this, first field is a number, 11 20 11 78 20

And I want to sort these lines according to the first field numerically , which means after sorting it should look like this, 11 11 20 20 78

but hadoop giving like this: 11 20 78

Upvotes: 1

Views: 1577

Answers (2)

Jijo
Jijo

Reputation: 610

That is because you are not iterating over the results. The reducer output will be a key and a list of values. To show all the values for a key you need to iterate over the list.

public void reducer(Text key ,Iterable<IntWritable> value, Context context) throws IOexception, InterruptedException{
    for(IntWritable number: value){
        context.write(key,number);
    }
}

Upvotes: 0

vefthym
vefthym

Reputation: 7462

The same key is handled by the same reducer. So, 11 and 11 are handled by the same reducer. If you output the key of this reducer, then 11 is output only once. The same goes for 20. If you just want to output 11 11 20 20 78, then do the following:

reduce(key, iterator values) {
    int counter = 0;
    while (values.hasNext()) {
        counter++;
        values.next();
    }
    for (int i =0; i < counter; ++i) {
        output.collect(key, <whatever>);
    }
}

Upvotes: 1

Related Questions