CODEWITHSUNDEEP
hadoopmapreducehadoop-yarnhadoop2mrv2
Praneeth
Praneeth

Reputation: 309

YARN MRv2 JobClient equivalent

I'm unable to find a JobClient (Java, MRv1) equivalent for MRv2. I'm trying to read MR job status, counters etc for a running job. I'd have to get the information from he resource manager I believe (since the History server wouldn't have the information before the job ends and I need to read counters while the job is still running). Is there a client in the mapreduce api that I'm missing?

Upvotes: 0

Views: 189

Answers (1)

Manjunath Ballur
Manjunath Ballur

Reputation: 6343

If you have the application ID of the MR job that you submitted to YARN, then you can use:

  • YarnClient (import org.apache.hadoop.yarn.client.api.YarnClient) and
  • ApplicationReport (import org.apache.hadoop.yarn.api.records.ApplicationReport)

to get application related statistics.

For e.g. sample code is below:

// Initialize and start YARN client
YarnClient yarnClient = YarnClient.createYarnClient();
yarnClient.init(configuration);
yarnClient.start();

// Get application report
try {
    ApplicationReport applicationReport = yarnClient.getApplicationReport(ConverterUtils.toApplicationId(applicationID));
    // Get whatever you want here.
} catch (Exception e) {
    // Handle exception;
}

// Stop YARN client
yarnClient.stop();

Some of the information you can get from the ApplicationReport class is:

  1. Application resource usage report

  2. Application dianostics

  3. Final application status

  4. Start and finish time

  5. Application type

  6. Priority

  7. Progress etc.

You can check the API documentation for YarnClient and ApplicationReport here (this is Hadoop 2.7 documentation):

Upvotes: 0

Related Questions