CODEWITHSUNDEEP
hadoophivebigdataoozie
Anup
Anup

Reputation: 955

Job queue for Hive action in oozie

I have a oozie workflow. I am submitting all the hive actions with 

<name>mapred.job.queue.name</name>
<value>${queueName}</value>

But for few hive actions, the job launched is not in specified queue; it is invoked in default queue.

Please suggest me the cause behind this behavior and solution.

Upvotes: 1

Views: 5267

Answers (3)

Mrin
Mrin

Reputation: 13

This is an extension to what Mathieu mentioned. It worked for me when I hard coded the queue name in below portion -

#in your hql file

SET tez.queue.name=queue name;

Upvotes: 0

Mathieu Barbot
Mathieu Barbot

Reputation: 173

In my case using Oozie it seems that there is no way to add --hiveconf tez.queue.name=myqueue in an xml tag. Trying also by updating the hive-site.xml on hdfs and still no usage of the defined queue. I've end up with updating the .hql file by adding the below at the beginning of the required query:

# in your hql file
SET tez.queue.name=${myqueue};

Upvotes: 0

Samson Scharfrichter
Samson Scharfrichter

Reputation: 9067

A. Oozie specifics Oozie propagates the "regular" Hadoop properties to a "regular" MapReduce Action.

But for other types of Action (Shell, Hive, Java, etc.) where Oozie runs a single Mapper task in YARN, it does not consider that it's a real MapReduce job. Hence it uses a different set of undocumented properties always prefixed with oozie.launcher. Look into that post for example.

So in your case the actual property to set would be oozie.launcher.mapred.job.queue.name

B. TEZ specifics Excerpt from HortonWorks documentation:

For example, in Hive you can use the tez.queue.name property in hive-site.xml to specify the queue to use for Hive-on-Tez jobs. To assign Hive-on-Tez jobs to use the "engineering" queue, add the following property to hive-site.xml:

<property>  <name>tez.queue.name</name> <value>engineering</value> </property>

Upvotes: 7

Related Questions