CODEWITHSUNDEEP
hadoop
superscalar
superscalar

Reputation: 33

How do I get counters in Hadoop

I'm using version 0.20.2 for the sake of MultipleOutputFormat,
and want to extract counters after the job is finished.

I tried to google some useful and practical examples, but I couldn't.
Anyone can show be a simple example?

Upvotes: 2

Views: 5069

Answers (1)

Thomas Jungblut
Thomas Jungblut

Reputation: 20969

If you are submitting your job like this:

  Configuration conf = new Configuration();
  Job job = new Job(conf);

  job.waitForCompletion(true);

And it has finished (you can call this even when its running, but the results won't be final then, because the job hasn't completed yet.), you can grab the counters by using:

long counter = job.getCounters().findCounter(ExplorationReducer.UpdateCounter.UPDATED)
    .getValue();

This is the name of the enum counter I used in my job:

ExplorationReducer.UpdateCounter.UPDATED

If you want all counter you have to traverse the backing structure behind the Counters object. There is an iterator for it.

Upvotes: 7

Related Questions