CODEWITHSUNDEEP
oozieoozie-coordinatoroozie-workflow
Yan
Yan

Reputation: 1

Oozie coordinator app, how to configure action triggered by external data source?

I would like to run a job every time when a external data source is updated, for example, some government file is update, http://www.ic.gc.ca/folder/filename.zip. Is there way of doing it?

Please provide some code examples with external URL data source.

Upvotes: 0

Views: 41

Answers (1)

ankush1377
ankush1377

Reputation: 208

Oozie coordinators are only meant to schedule workflows based on a specific frequency. Coordinators do not support workflow executions based on configurable actions.

In case you have such a requirement and have to use oozie, I would suggest 2 possible ways:-

  1. Run your coordinator with a very small frequency (as compared to the frequency of your file update) and configure an initial action in your workflow to check if the file has been updated. In case it has, move to the action that you want to perform, otherwise just end the workflow (this however doesen't truly remain a synchronous process)
  2. Do not use coordinators. Just configure an oozie workflow with the desired actions & develop a process to trigger that workflow as & when your file is updated. Please note that the process to trigger will have to be a solution external to Oozie (bash script, API, queue message etc.)

Upvotes: 0

Related Questions