CODEWITHSUNDEEP
hadoop
user3782579
user3782579

Reputation: 315

Hadoop multiple datanodes on single machine

I'm interested in configuring multiple data nodes (say 2) on a single Linux machine.

I made some research and followed a tutorial but didn't work.

If someone can give a working tutorial or show me how to do it.

Thanks

Upvotes: 1

Views: 3215

Answers (1)

SachinJose
SachinJose

Reputation: 8522

Instead of creating separate VM's you can run multiple datanode instance in the same node itself. Only thing is you got to maintain different configurations for each datanode instance.

Need to specify different port for each datanode daemons override the following parameters in hdfs-site.xml:

dfs.datanode.address  0.0.0.0:50010  The datanode server address and port for data transfer.
dfs.datanode.http.address    0.0.0.0:50075  The datanode http server address and port.
dfs.datanode.ipc.address     0.0.0.0:50020  The datanode ipc server address and port.

Also need to maintain different data directory for each datanode daemons override the following parameters

dfs.datanode.data.dir

Upvotes: 1

Related Questions