CODEWITHSUNDEEP
hadoopjob-scheduling
MRK
MRK

Reputation: 21

Hadoop Job Scheduling query

I am a beginner to Hadoop.

As per my understanding, Hadoop framework runs the Jobs in FIFO order (default scheduling).

Is there any way to tell the framework to run the job at a particular time?

i.e Is there any way to configure to run the job daily at 3PM like that?

Any inputs on this greatly appreciated.

Thanks, R

Upvotes: 1

Views: 1485

Answers (4)

scott
scott

Reputation: 26

I'd use a commercial scheduling app if Cron does not cut it and/or a custom workflow solution. We use a solution called jams but keep in mind it's .net-oriented.

Upvotes: 0

Pranab
Pranab

Reputation: 663

How about having a script to execute your Hadoop job and then using at command to execute at some specified time.if you want the job to run regularly, you could setup a cron job to execute your script.

Upvotes: 0

wlk
wlk

Reputation: 5785

you might consider using Oozie (http://yahoo.github.com/oozie/). It allows (beside other things):

Frequency execution: Oozie workflow specification supports both data and time triggers. Users can specify execution frequency and can wait for data arrival to trigger an action in the workflow.

It is independent of any other Hadoop schedulers and should work with any of them, so probably nothing in you Hadoop configuration will change.

Upvotes: 4

Ivan
Ivan

Reputation: 776

What about calling the job from external java schedule framework, like Quartz? Then you can run the job as you want.

Upvotes: 4

Related Questions