CODEWITHSUNDEEP
hadoopoozie
user2861483
user2861483

Reputation: 11

How to Remove hardcode values(like cache file) from workflow.xml in oozie and make it dynamic?

I have hardcode values in workflow.xml file(like cache file) Workflow.xml file contains follwoing jar as hardcoded

archivedir/tutorial-udf.jar#udfjar

I want to remove tutorial-udf.jar hardcoded value and make it dynamic. one way i can do this by writting the hardcoded values in to the job.properies file and passing it to workflow.xml file. Like below code in workflow.xml

/${tutorial-udf}#udfjar

This is one way. Please suugest another way of doing this.

Technology oozie , Hadoop

Upvotes: 0

Views: 419

Answers (1)

Cipi
Cipi

Reputation: 160

If I understand correctly, you don't want to hardcode the name of the jar file in the job files (workflow.xml or job.properties) and you search for a way to determine the name of the jar at runtime. This way the jar can be changed between 2 successive runs of the same job, without the need of updating any of the job files.
If my understanding is correct, you could have a java action that "computes" the name of the jar. That computation can be various: the existence of a jar in a predefined HDFS folder, the newest jar in a predefined HDFS folder, the name of the jar written in a predefined file in HDFS, etc.
In this case:

1. In the java class you "save" the jar name:

Properties props = new Properties();
props.setProperty("jarFileName", jarName);
OutputStream os = new FileOutputStream(new File(System.getProperty("oozie.action.output.properties")));
props.store(os, "");
os.close();


2. In workflow.xml you specify that the java class produces properties (the <capture-output/> tag):

<action name="init-jar-name">
    <java>
       <!-- Configurations -->
       <main-class>com.example.JarNameFinder</main-class> 
       <capture-output /> <!--  Important for retrieving the properties set in the java class --> 
    </java>
    <ok to="nextNodeName" />
    <error to="errorNodeName" /> 
</action>


3. In workflow.xml, in the place you need the jar file name, you specify it like this:

${wf:actionData('init-jar-name')['jarFileName']}

Upvotes: 2

Related Questions