CODEWITHSUNDEEP
hadoophdfs
Liondancer
Liondancer

Reputation: 16469

using hdfs dfs -test to see if directory exists

In the hadoop docs:

test

Usage: hadoop fs -test -[defsz] URI

Options:

-d: f the path is a directory, return 0.
-e: if the path exists, return 0.
-f: if the path is a file, return 0.
-s: if the path is not empty, return 0.
-z: if the file is zero length, return 0.
Example:

hadoop fs -test -e filename

I want to do something if the hdfs directory does not exist. Every argument in the -test option returns 0. How can I can an output if a directory does NOT exist? 

drwx------   - bli1 hadoop_bips          0 2016-01-29 12:48 /user/bli1/.Trash
drwx------   - bli1 hadoop_bips          0 2016-01-08 09:41 /user/bli1/.staging
drwxr-x---   - bli1 hadoop_bips          0 2015-03-31 09:35 /user/bli1/camus_test
drwxr-x---   - bli1 hadoop_bips          0 2015-11-18 12:41 /user/bli1/hdfs_archive
drwxr-x---   - bli1 hadoop_bips          0 2016-01-26 16:18 /user/bli1/hdfs_archive_archive
drwxr-x---   - bli1 hadoop_bips          0 2016-01-26 16:19 /user/bli1/hdfs_archive_concatenate
drwxr-x---   - bli1 hadoop_bips          0 2016-01-26 16:15 /user/bli1/hdfs_archive_delete
drwxr-x---   - bli1 hadoop_bips          0 2015-03-31 16:20 /user/bli1/output
drwxr-x---   - bli1 hadoop_bips          0 2015-03-29 18:09 /user/bli1/wordcount

I tried this

$ hdfs dfs -test -d /user/bli1/testdir     # nothing is returned
$
$ hdfs dfs -test -d /user/bli1/output      # nothing is returned

I am expecting a 0? I see no 0. I don't notice anything different between the 2 commands.

Upvotes: 3

Views: 14425

Answers (1)

Ana
Ana

Reputation: 831

That is what "$?" does. It stores the status of last executed command. You can check its value. If it is 0, then, bingo! 

target_dir=<dir in hdfs>

hadoop fs -test -d ${target_dir};

if [ $? -eq 0 ]
then 
    echo "Directory exists!"
else
    echo "Directory does not exists!" 
fi

Upvotes: 5

Related Questions