CODEWITHSUNDEEP
oozieoozie-coordinator
user1122
user1122

Reputation: 307

can we create dynamic directory structure in oozie coordinator?

I want to create an event driven oozie coordinator. but the directory path changes regularly. I don't want to hard code the directory in the code.

    <datasets>
        <dataset name="test_co" frequency="${coord:minutes(120)}" initial-instance="${coordStartDate}" timezone="${timezone}">
            <uri-template>**${nameNode}/dynamicName**</uri-template>
            <done-flag>_OK</done-flag>
        </dataset>
    </datasets>

How can i run shell script before this action is triggered it creates the folder name and check if OK file is present inside that folder or not?

Upvotes: 1

Views: 277

Answers (1)

Abhijit Mehetre
Abhijit Mehetre

Reputation: 53

Oozie supports creating dynamic directory structure i.e. dated directories using coordinator datasets (if possible use with ). e.g.

<datasets>  
     <dataset name="logs" frequency="${coord:hours(1)}" initial-instance="2009-       01-01T01:00Z" timezone="UTC">
    <uri-template>hdfs://bar:9000/app/logs/${YEAR}${MONTH}/${DAY}/${HOUR}</uri-template>
  </dataset>
</datasets>

After running above oozie code today viz.22-03-2017 16:00 PM The directory structure would be like : hdfs://bar:9000/app/logs/2017/03/22/16

Upvotes: 1

Related Questions