CODEWITHSUNDEEP
hadoopoozie
Jasjyot Singh Jaswal
Jasjyot Singh Jaswal

Reputation: 31

Output events not being generated by oozie coordinator

My coordinator app is successfully reading from the input events and executing the workflow at the nominal time, however output events are not being generated at all.

I tried setting the output event with without a done flag, but it did not work. What have I done wrong?

Below is my coordinator.xml file.

 <coordinator-app name="wf_scheduler" frequency="${coord:days(1)}"
    start="2016-12-14T00:00Z" end="2016-12-19T00:00Z" timezone="IST"
    xmlns="uri:oozie:coordinator:0.2">
    <controls>
        <timeout>30</timeout>
        <concurrency>3</concurrency>
        <execution>FIFO</execution>
        <throttle>3</throttle>
    </controls>
    <datasets>
        <dataset name="inp_logs" frequency="${coord:days(1)}"
            initial-instance="2016-12-01T00:00Z" timezone="IST">
            <uri-template>
                /user/cloudera/inp_logs/${YEAR}_${MONTH}_${DAY}
            </uri-template>
            <done-flag></done-flag>
        </dataset>
        <dataset name="opt_logs" frequency="${coord:days(1)}"
            initial-instance="2016-12-01T00:00Z" timezone="IST">
            <uri-template>
                /user/cloudera/opt_logs/${YEAR}_${MONTH}_${DAY}
            </uri-template>
            <done-flag>_trigger</done-flag>
        </dataset>
    </datasets>
    <input-events>
        <data-in name="input" dataset="inp_logs">
            <instance>${coord:current(0)}</instance>
        </data-in>
    </input-events>
    <output-events>
        <data-out name="output" dataset="opt_logs">
            <instance>${coord:current(0)}</instance>
        </data-out>
    </output-events>
    <action>
        <workflow>
            <app-path>${app_path}</app-path>
        </workflow>
    </action>
</coordinator-app>

Upvotes: 2

Views: 606

Answers (1)

la_femme_it
la_femme_it

Reputation: 672

Well, I have the same issue. I even set empty . I created manually _SUCCESS file. I dig up. Found in book Apache Oozie: The Workflow Scheduler for Hadoop. But I have no idea, if it is valid.

No, files specified in output-events are not automatically created by Oozie, you need to create those files in your Oozie workflow actions.

The output-events configuration is for giving Oozie information on what files will be created by your Oozie workflow actions, which Oozie would use to cleanup files when rerunning/reprocessing a coordinator.

Upvotes: 1

Related Questions