CODEWITHSUNDEEP
hadoopmapreduce
ahmadou_20
ahmadou_20

Reputation: 21

Output.collect mapreduce does not seem to be taking the right value?

I'm quite new to Hadoop and I've been struggling for two days to figure out why output.collect is not collecting the right value.

I explain myself: in fact, (for sake of simplification) I have the following map method :

  public
      void
      map(LongWritable key, Text value, OutputCollector<Text, MyObject> output, Reporter reporter)
                                                                                                                 throws IOException {

try {

  ForXmlHandling message = (ForXmlHandling) unmarshaller.unmarshal(new StringReader(value.toString()));

  MyObject row = XmlParser.parse(message);

   row.setOrigin(true);
  output.collect(new Text(row.getPnrRecordKey().toString()), row);
  }

catch(JAXBException e) {
   LOG.debug(e);
}

}

where MyObject is an object I created:

  public class MyObject  {

  private boolean                   original;
  private boolean                   split;
  ....

  }

In fact, when I launch only the mapper in the debug mode, even despite the fact I'm setting the origin attribute of row (MyObject) to true, the output of the mapper (output.collect) is always row with origin attribute set to false (default value of a boolean). I do not understand what is wrong with output.collect.

Any help would be more than welcome. Thanks!

Upvotes: 0

Views: 41

Answers (1)

ahmadou_20
ahmadou_20

Reputation: 21

Thanks for your answer Matt ! Indeed, the problem was coming from the implementations of readFields and write as I was not calling: 

//write
_original.write(out);
_split.write(out);


//readFields
_original = new BooleanWritable();
_split = new BooleanWritable();

_original.readFields(in);
_split.readFields(in);

Upvotes: 1

Related Questions