CODEWITHSUNDEEP
javahadoopoozieoozie-coordinatorapache-crunch
Tanveer Dayan
Tanveer Dayan

Reputation: 506

Not able to set mapred.job.queue.name in Oozie java action

I have an application which runs crunch jobs. I am trying to configure Oozie to run this job using a java action. My action is as given below,

    <workflow-app name="Wworkflow" xmlns="uri:oozie:workflow:0.4">
<start to="TestWw"/>
<action name="TestWw">
    <java>
        <job-tracker>${jobTracker}</job-tracker>
        <name-node>${nameNode}</name-node>
        <configuration>
            <property>
                <name>mapred.job.queue.name</name>
                <value>launcher</value>
            </property>
        </configuration>
        <main-class>com.test.Main</main-class>
        <java-opts>-Dmapred.output.compress=false -Dmapred.textoutputformat.separator=, -Dcrunch.disable.output.counters=true</java-opts>

    </java>
    <ok to="end"/>
    <error to="kill"/>
</action>
<kill name="kill">
    <message>Action failed, error message[${wf:errorMessage(wf:lastErrorNode())}]</message>
</kill>
<end name="end"/>

But everytime I run the Oozie job, my job throws

     org.apache.hadoop.ipc.RemoteException(java.io.IOException): Queue "default" does not exist

I have configured the queue to be launcher, but oozie just doesnt use this attribute.

Please note that I am running crunch jobs which are just internal map reduce jobs.

Upvotes: 2

Views: 5262

Answers (1)

Ben Watson
Ben Watson

Reputation: 5521

When an Oozie action starts, it creates a launcher application in YARN which then kicks off the job itself. The launcher can run in a different queue to the action itself, and the queue in which the launcher runs is actually set with oozie.launcher.mapred.job.queue.name. I suspect that what's happening is that your launcher is trying to run in default, and so the Crunch job itself never actually kicks off.

Setting this should solve your problem:

<property>
       <name>oozie.launcher.mapred.job.queue.name</name>
       <value>launcher</value>
</property>

As an aside, mapred.job.queue.name has been deprecated - use mapreduce.job.queuename.

Upvotes: 4

Related Questions