CODEWITHSUNDEEP
hadoop
shakti
shakti

Reputation: 23

I am getting errors while copying files from local to hdfs

I am getting error while copying files from local file system to hdfs,
will you please help me regarding this,
I am using this command :

hadoopd fs -put text.txt file

Upvotes: 0

Views: 762

Answers (5)

sathish
sathish

Reputation: 35

"DataStreamer Exception: org.apache.hadoop.ipc.RemoteException: java.io.IOException: File /user/hduser/myfile could only be replicated to 0 nodes, instead of 1 at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getAdditionalBlock". From this I thinrk your data node is not running/properly. Check that in cluster UI.Then try

hadoop dfs -put /path/file /hdfs/file (hadoop YARN) hadoop fs -copyFromLocal /path/file /hdfs/file (hadoop1.x)

Upvotes: 0

Jaraxal
Jaraxal

Reputation: 73

Without knowing the specific error you are getting, it's difficult to answer. The other responders posted the proper syntax. However, it is not uncommon to see permission issues when attempting to copy files to HDFS.

By default the user and group are typically "hdfs" and "supergroup". Your user account likely doesn't belong to "supergroup" and will get permission denied errors. Try running the command as:

sudo -u hdfs hadoop fs -put /path/to/local/file /path/to/hdfs/file

or

sudo -u hdfs hadoop dfs -put /path/to/local/file /path/to/hdfs/file

You can get around having to do this by changing the ownership and permission of the destination directory on HDFS to be more permissive.

Upvotes: 0

USB
USB

Reputation: 6139

put and copyFromLocal command helps you to copy data from your local system to HDFS,provided you have the permission to do so.

hadoop fs -put /path/to/textfile /path/to/hdfs
OR
hadoop dfs -put /path/to/textfile /path/to/hdfs

Comming to your error:

You typed the above command as

hadoopd fs

Upvotes: 2

Shekhar
Shekhar

Reputation: 11788

You can use following command

hadoop fs -copyFromLocal text.txt <path_to_hdfs_directory_where_you_want_to_keep_text.txt>

Upvotes: 0

Y.Prithvi
Y.Prithvi

Reputation: 1221

use

hadoop dfs -put /text.txt /file

hadoop dfs -put /path/to/local/file /path/to/hdfs/file

Upvotes: 0

Related Questions