CODEWITHSUNDEEP
javahadoop
Karthik. V
Karthik. V

Reputation: 33

How to stop a Hadoop job from within Java

I have an server setup that consists of 8 machines on which I run a Hadoop job to download certain assets. A client side agent uses JobConf to start the Hadoop jobs.

The agent gets a response code from the server based on which it can determine how the download proceeded. In the below code, invoke() creates a connection to the REST API exposed by the server. ResponseData is a custom class that can read the downloaded data off the connection associated with the response as below: 

ResponseData res = invoke(downloadUrl, contentType);
downloadedAssetStream = New ObjectInputStream(res.connection.getInputStream);

if(res.code != 200)
{
    //Stop hadoop job
}

For stopping the job, what do I need? I've heard somewhere that throwing an IOException would do, but I haven't been able to verify it yet.

Upvotes: 3

Views: 3255

Answers (1)

Praveen Sripati
Praveen Sripati

Reputation: 33495

Job#killJob with the new MR API or RunningJob#killJob with the old MR API will kill the Job.

Upvotes: 1

Related Questions