CODEWITHSUNDEEP
debianooziegoogle-cloud-dataproc
Revan
Revan

Reputation: 551

Dataproc failing to install oozie package

I am trying below command on google cloud dataproc:

apt-get install oozie-client

But dataproc is failing to list oozie package in repository.

error:

root@<master>:#> apt-get install oozie-client
Reading package lists... Done
Building dependency tree
Reading state information... Done
E: Unable to locate package oozie-client

Does any one know how to fix this issue?

Upvotes: 1

Views: 387

Answers (1)

James
James

Reputation: 2331

The answer for this is probably best said it's complicated.

oozie-client does not currently compile against Hive 2 and Spark 2. Newer versions of Cloud Dataproc are built with Spark 2 and Hive so. Therefore, at this point, the oozie-client is excluded in our build because it's a bit messy.

It might work with Pig and MapReduce, though. So, it depends on your use case. Are you planning on using it with Pig/MR or Hive/or/Spark?

Apache Airflow has been increasingly used with Cloud Dataproc and seems to work pretty well. Of course, there are some tradeoffs there, such as the fact that Airflow is probably best seperate from the cluster itself, can take awhile to install, and is an incubating project.

(Source - I am the Cloud Dataproc PM)

Upvotes: 1

Related Questions