CODEWITHSUNDEEP
javahadoopapache-pigoozie
user1945134
user1945134

Reputation: 1

Async pig job submission

I am dynamically generating some Pig query in Java and executing the same using PigServer Class. Is there any way in which I can submit the job asynchronously and later poll/hook for job completion? Also, if pig doesn't allow it , can I use Oozie with Java Action to achieve the desired behaviour.

Upvotes: 0

Views: 382

Answers (2)

Chris White
Chris White

Reputation: 30089

You can use the fork and join control nodes in OOZIE to run jobs in parallel.

Executing dynamic code is easy if you just want to change some properties of the Pig script to run (you can use template variables - see the oozie pig action docs). However if by dynamic you mean dynamically building different code paths then you'll need a custom Java action to write out the pig script to HDFS before calling the pig action.

Upvotes: 1

Shantha Kumar
Shantha Kumar

Reputation: 1

First post your query properly.The above description is not well explanatory.If you want to run any job asynchronously in java ,you should use Thread programming.There is a good apache framework called "quartz-scheduler" which can do this task.It is open source.Refer this link.

http://quartz-scheduler.org/documentation/quartz-2.x/tutorials/

Upvotes: 0

Related Questions