CODEWITHSUNDEEP
hadoopdocker-compose
Pavan Keerthi
Pavan Keerthi

Reputation: 1681

Hadoop fs -put has DataStreamer Exception java.nio.channels.UnresolvedAddressException

I have started the following docker containers

namenode  
datanode1  
datanode2  
datanode3

When I attempt to put a file into HDFS as follows I get this error

hdfs dfs -put /tmp/airline-2008.csv.bz2 hdfs://namenode.bigdatacluster.com:8020/data/

17/09/28 22:49:01 WARN hdfs.DataStreamer: DataStreamer Exception java.nio.channels.UnresolvedAddressException at sun.nio.ch.Net.checkAddress(Net.java:101) at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:622) at org.apache.hadoop.net.SocketIOWithTimeout.connect(SocketIOWithTimeout.java:192) at org.apache.hadoop.net.NetUtils.connect(NetUtils.java:531) at org.apache.hadoop.hdfs.DataStreamer.createSocketForPipeline(DataStreamer.java:226) at org.apache.hadoop.hdfs.DataStreamer.createBlockOutputStream(DataStreamer.java:1591) at org.apache.hadoop.hdfs.DataStreamer.nextBlockOutputStream(DataStreamer.java:1547) at org.apache.hadoop.hdfs.DataStreamer.run(DataStreamer.java:658) put: java.nio.channels.UnresolvedAddressException

Upvotes: 0

Views: 3036

Answers (1)

Shubhangi
Shubhangi

Reputation: 2254

From Oracle java doc, 

Unchecked exception thrown when an attempt is made to invoke a network operation upon an unresolved socket address

That means hostname namenode.bigdatacluster.com used in hdfs URL is not getting resolved to IP address. You may resolve this by adding host to IP mapping to /etc/hosts file.

Upvotes: 2

Related Questions