CODEWITHSUNDEEP
hadoopcloudera-cdh
tuk
tuk

Reputation: 6842

Updating individual CDH Components in a Community Edition via '1 Click Installer'

Can someone let me know if it possible to update individual CDH component to 5.13 from 5.7 via "1 Click Installer" for Community Edition?

For example, let's say I want to update only the hadoop-hdfs-datanode to the latest in a server. If I do sudo apt-get install hadoop-hdfs-datanode it is updating other CDH component also running in that node (like resource-manager, node-manager, etc).

As discussed here if I am trying to upgrade hadoop-yarn-resourcemanager it is upgrading almost all the cdh hadoop components

support@platform1:~$ sudo apt-get install hadoop-yarn-resourcemanager
Reading package lists... Done
Building dependency tree
Reading state information... Done
The following extra packages will be installed:
  hadoop hadoop-0.20-mapreduce hadoop-client hadoop-conf-pseudo hadoop-hdfs
  hadoop-hdfs-datanode hadoop-hdfs-journalnode hadoop-hdfs-namenode
  hadoop-hdfs-secondarynamenode hadoop-hdfs-zkfc hadoop-mapreduce
  hadoop-mapreduce-historyserver hadoop-yarn hadoop-yarn-nodemanager
The following packages will be upgraded:
  hadoop hadoop-0.20-mapreduce hadoop-client hadoop-conf-pseudo hadoop-hdfs
  hadoop-hdfs-datanode hadoop-hdfs-journalnode hadoop-hdfs-namenode
  hadoop-hdfs-secondarynamenode hadoop-hdfs-zkfc hadoop-mapreduce
  hadoop-mapreduce-historyserver hadoop-yarn hadoop-yarn-nodemanager
  hadoop-yarn-resourcemanager
15 upgraded, 0 newly installed, 0 to remove and 16 not upgraded.

Upvotes: 0

Views: 58

Answers (1)

OneCricketeer
OneCricketeer

Reputation: 191681

it is updating other CDH component also running in that node

I doubt it is upgrading everything in the node, just the dependent services of upgrading the hadoop client.

If you were to install Hadoop all by itself, it includes HDFS, MapReduce, YARN, and the Hadoop client libraries. Therefore, it makes sense that upgrading the datanode package would try to grab those, but not HBase, Hive, Pig, Spark, Oozie, etc. packages.

Essentially, you need to ensure all your Hadoop client libraries are the same version. CDH itself hasn't moved off of Hadoop 2.6.0 between those releases, although it has added patches to that base release, so it might be fine to upgrade.

However, let's take HBase as an example. From the documentation, it says Hadoop 2.6.0, 2.7.0 nor Hadoop 2.8.x are supported; Hadoop 3.x is not tested; only 2.6.1+ or 2.7.1+ are supported.

And continues on to say that

In distributed mode, it is critical that the version of Hadoop that is out on your cluster match what is under HBase... Make sure you replace the jar in HBase across your whole cluster. Hadoop version mismatch issues have various manifestations but often all look like its hung

All component upgrades should be followed through, and Cloudera makes the effort to ensure all components of a single release work together, not mixed across releases.

Upvotes: 0

Related Questions