CODEWITHSUNDEEP
hadoophdfs
Felice Pollano
Felice Pollano

Reputation: 33252

Moving hadoop master node in another box: how to handle HDFS

I've a small hadoop cluster with a master node and a slave, and for some reason the master box has to be moved on another machine. While I can manage to reconfigure the node quite easily, and I have a strategy to migrate the hive metadata and other stuff, my concern is about HDFS. Do I have to move the file of the hadoop_tmp ( where my hdfs is located ) on the target machine? Or the cluster will reconstruct by replicating files from the server? My fear is to land with an inconsistent hdfs, what is the best?

Upvotes: 0

Views: 476

Answers (2)

Felice Pollano
Felice Pollano

Reputation: 33252

I describe here how I did it, since it worked, do't know if it is the best way, but it works without having file system in an inconsistent state. Very simple approach was:

  1. set HDFS safe mode: hdfs dfsadmin -safemode enter
  2. stop the cluster
  3. configure the new master with all hadoop required software, users and configurations

  4. copy files from the hadoop file system on the new master machine, changing permissions accordingly

  5. change the DNS to reflect the new topology (maybe /etc/hosts ),or change the IP when needed in all cluster config file to reflect new master address

  6. start the new cluster, it should start as usual, starting even the slaves

  7. check DFS, and exit safe mode: hdfs dfsadmin safemode -leave

Upvotes: 0

Kassav'
Kassav'

Reputation: 1150

DistCP would be the best approch. DistCP

Upvotes: 1

Related Questions