CODEWITHSUNDEEP
hadoopjenkinsjenkins-pluginsoozie
Knows Not Much
Knows Not Much

Reputation: 31546

Deploy Oozie jobs via Jenkins

I am writing Oozie jobs and I want to deploy them to a hadoop cluster.

However rather than doing them manually, I want to use Jenkins to do this deployment for me.

I searched the web and didn't find any plugin for oozie .... so if you are using oozie on a hadoop cluster, can you please tell me what is the best way in which I could do automated build & deployments using Jenkins?

Upvotes: 3

Views: 3327

Answers (1)

Łukasz Gawron
Łukasz Gawron

Reputation: 917

Deployment of Oozie workflows is just putting xml files and required dependencies not available on the cluster to HDFS directory. link

  • Assuming that you have two Jenkins jobs,
    • one for build, test etc
    • second for deployment which is triggered after sucesful build of first job

you can script putting files on HDFS in this second job for example using WebHdfs

Other way if product of your build from first job is some kind of a package e.g. zip, you can create a script which will download this artifact from Jenkins/Nexus to machine with access to hdfs console command, extract this zip and just put whole directory to hdfs, as its described here

If you are using some provisioning tool (Puppet or Ansible) you can use it for this purpose.

For configuring Jenkins scheduling you may want to use

  • Jenkins Pipeline Plugin
  • or on Jenkins Job page "Configure -> Build Triggers -> Build after other projects are built"

Upvotes: 5

Related Questions