CODEWITHSUNDEEP
hadoophdfs
Foxlooo
Foxlooo

Reputation: 69

No such file or directory with hdfs

When I do

$ ls

I find a file with a name: file_name.csv

However, when I try to do 

$ hdfs dfs -put /home/user_name/file_name.csv tempfolder

(/home/user_name/ is where file_name.csv is located while tempfolder is a tempfolder I created in hdfs). It says

"`/home/user_name/file_name.csv': No such file or directory"

even though it is there. Is there a reason behind this or does anyone know how to fix it?

Note: If this is relevant, when I call hdfs dfs..., it is with a sudo. So

sudoing "hdfs dfs -put /home/user_name/file_name.csv tempfolder"

Upvotes: 0

Views: 5085

Answers (3)

Rudra
Rudra

Reputation: 243

The case may arise if the file path is long which was in my case . For eg.

hdfs dfs -put /home/user/folder1/folder2/folder3/demo.csv /usr/input

Instead Navigate to the nearest directory then,

hdfs dfs -put /folder3/demo.csv /usr/input

Upvotes: 0

OneCricketeer
OneCricketeer

Reputation: 191671

First, '/home/user_name/file_name.csv': No such file or directory" is an error from your local machine. If you see the file with ls, then do $PWD/file_name.csv

Sudo has very little to do with the problem.

There is a difference between 

 hdfs:///tempfolder
 hdfs:///user/username/tempfolder

And if you sudo, you're running the commands as a different user account, not yourself.

The error should be the same for 

hdfs dfs -ls /user/username/tempfolder

You need a user directory if you don't give an absolute path. In other words, hdfs commands always assume you're writing to your HDFS user folder by default.

while tempfolder is a tempfolder I created in hdfs

Created how?

Using sudo? Because then the directory won't exist for your user account in HDFS.

Did you create /tempfolder? That leading slash is very important and needs to be maintained in your put command

If that HDFS folder didn't already exist, a trailing slash is also important - otherwise, you're copying your CSV to a file without extension named tempfolder

Upvotes: 1

Mike D
Mike D

Reputation: 64

Might be running into a shell expansion. Try a full path to the destination:

hdfs dfs -put /home/user_name/file_name.csv /tempfolder

Upvotes: 0

Related Questions