CODEWITHSUNDEEP
javamapreduce
testAccount
testAccount

Reputation: 25

Mapreduce function to calculate in degree and out degree and show sum using java

I am trying to sum the in degree and out degree for a set of data. Here is the sample data:

Source  Target

1        2  
2        1  
3        1  
2        3

So the expected output is:

ID     In degree   Out degree  
1       2            1  
2       1            2  
3       1            1

How can this be achieved with mapreduce Java and print out the results in a single line.

Upvotes: 1

Views: 1743

Answers (1)

mangusta
mangusta

Reputation: 3544

One option involving one MR job: Assuming that the original dataset looks like [node1,node2]:

-mapper reads original dataset and emits triples [node1,out] and [node2,in] for every line

-reducer gets triples from mapper in form of [key,label], computes the outdegree and indegree by counting "out" labels and "in" labels separately per key and outputs them in form of [key, indegree, outdegree]

Implementation would look similar to below (assuming that node1 and node2 in your dataset are separated by space and also assuming that the dataset contains distinct pairs only):

Mapper:

public class YourMapper extends MapReduceBase implements Mapper<LongWritable, Text, Text, Text> {

      public void map(LongWritable key, Text value, OutputCollector<Text, Text> output, Reporter reporter) throws IOException {

        String line = value.toString();     
        String[] line_spl = line.split(" ");

        String node1 = line_spl[0];
        String node2 = line_spl[1];

        Text node1_txt = new Text(node1);
        Text node2_txt = new Text(node2);
        Text emit_out = new Text("out");
        Text emit_in  = new Text("in");

        output.collect(node1_txt, emit_out);
        output.collect(node2_txt, emit_in );

      }//end map function


}//end mapper class

Reducer: 

public class YourReducer extends MapReduceBase implements Reducer<Text, Text, Text, Text> {

    public void reduce(Text key, Iterator<Text> values, OutputCollector<Text, Text> output, Reporter reporter) throws IOException {

         int count_outs = 0;
         int count_ins  = 0;

            while (values.hasNext()) {

              Text value = (Text) values.next();

              String value_str = value.toString();

              if(value_str.equals("out"))
                 count_outs++;
              else
              if(value_str.equals("in"))
                 count_ins++;  

            }

            Text out = new Text(count_ins + " " + count_outs);              
            output.collect(key, out);

    }//end reduce function

}//end reducer class

Upvotes: 2

Related Questions