CODEWITHSUNDEEP
bashhadoop
gjin
gjin

Reputation: 929

No error when creating a directory, but no directory was created

start.sh starts copy_file.sh and passes it 2 parameters

yeasterday_with_dash=2017-01-31
today_without_dash=20170201

echo "-----------RUN copy mta-------------"
bash copy_file.sh mta $today_without_dash
echo "-----------RUN copy rcr-------------"
bash copy_file.sh rcr $today_without_dash
echo "-----------RUN copy sub-------------"
bash copy_file.sh sub $today_without_dash

copy_file.sh is supoosed to check if a directory exists and if not create it.

hdfs_dir=/apps/hive/warehouse/mydb.db/fct_evkuzmin/${2}/file_${1}

if hadoop fs -test -d $hdfs_dir ; then
    echo "Directory "$hdfs_dir" exists"
else
    hadoop fs -mkdir $hdfs_dir
    echo "Creating "$hdfs_dir
fi

The script runs succesfully it seems.

The log.

yeasterday_with_dash=2017-01-31
today_without_dash=20170201
-----------RUN copy mta-------------
mkdir: `/apps/hive/warehouse/mydb.db/fct_evkuzmin/20170201/file_mta': No such file or directory
Creating /apps/hive/warehouse/mydb.db/fct_evkuzmin/20170201/file_mta
end copy mta files
-----------RUN copy rcr-------------
mkdir: `/apps/hive/warehouse/mydb.db/fct_evkuzmin/20170201/file_rcr': No such file or directory
Creating /apps/hive/warehouse/mydb.db/fct_evkuzmin/20170201/file_rcr
end copy rcr files
-----------RUN copy sub-------------
mkdir: `/apps/hive/warehouse/mydb.db/fct_evkuzmin/20170201/file_sub': No such file or directory
Creating /apps/hive/warehouse/mydb.db/fct_evkuzmin/20170201/file_sub
end copy sub files

But there is no actual directory created. What's the problem?

Upvotes: 0

Views: 32

Answers (1)

user3560058
user3560058

Reputation: 36

Use mkdir with -p argument to also create parent directories.

hadoop fs -mkdir -p etc

Upvotes: 2

Related Questions