bash tar error doesn't create tar.gz

Question

I have the following bash script:

#DIR is something like: /home/foo/foobar/test/ without any whitespace but can also include whitespace
DIR="$( cd "$( dirname "$0" )" && pwd )"
#backup_name is read from a file
backup_name=FOOBAR
date=`date +%Y%m%d_%H%M_%S`
#subdirs is also read from the same file
subdirs=etc/ sbin/ bin/
filename="$DIR/Backup_$backup_name"_"$date.tar.gz"

cd /
echo "filename: $filename"
echo "subdirs $subdirs"
cmd='tar czvf "'$filename'" '$subdirs
echo "cmd tar: $cmd"
$cmd

But I get following output:

filename: /home/foo/foobar/test/Backup_FOOBAR_20120322_1529_35.tar.gz
subdirs: etc/ sbin/ bin/
cmd tar: tar cfvz "/home/foo/foobar/test/Backup_FOOBAR_20120322_1529_35.tar.gz" etc/ sbin/ bin/
etc/
# ... list of files in etc
# but no files from sbin or bin directory
tar: "/home/foo/foobar/test/Backup_FOOBAR_20120322_1529_35.tar.gz": can open not execute: File or directory not found
tar: not recoverable error: abortion.

However, when I copy the echo output of the tar command, make a cd to / and paste it into the bash shell it is working:

tar cfvz "/home/foo/foobar/test/Backup_FOOBAR_20120322_1529_35.tar.gz" etc/ sbin/ bin/
etc/

• Every variable is defined and there is no trailing newline
• I also tried $cmd with backticks
• the two variables: backup_name and subdirs are read from a file (I did not include the reading process in the code)

edit: I just copied my script to a dir with no whitespace and changed the line:

cmd='tar czvf "'$filename'" '$subdirs
#to
cmd="tar czvf $filename $subdirs"

and it's working now but when I do the same in a dir which also contents whitespaces I get still the same error.

edit2: reading from file (the file is read before anything else happens)

config="config.txt"
local line
while read line
do
    #points to next free element and declares it
    config_lines[${#config_lines[@]}]=$line
done <$config
backup_name=${config_line[0]}
subdirs=${config_line[1]}

What is wrong with my bash script?

Answer 1

Short answer: see BashFAQ #050: I'm trying to put a command in a variable, but the complex cases always fail!.

Long answer: embedding quotes in a variable doesn't do anything useful, because when you use it (i.e. $cmd), bash parses quotes before replacing variables; by the time the quotes are there, it's too late for them to do any good. You do, however, have several options:

1. Don't bother with putting the command in a variable in the first place, just use it directly:

   echo "filename: $filename"
   echo "subdirs $subdirs"
   tar czvf "$filename" $subdirs
   

2. If you really need to put it in a variable first, use an array rather than a plain text variable (and ideally, do the same with the subdirs list):

   subdirs=(etc/ sbin/ bin/)
   ...
   
   echo "filename: $filename"
   echo "subdirs ${subdirs[*]}"
   cmd=(tar czvf "$filename" "${subdirs[@]}")
   printf "cmd tar:"
   printf " %q" "${cmd[@]}" # Have to do some trickery to get it printed right
   printf "\n"
   "${cmd[@]}"
   

Answer 2

OK, your original script did not work because file/path determination happens before variable expansion, so the filename is wrong: tar thinks that it's supposed to write to a file in the current directory named "/home/foo/foobar/test/Backup_FOOBAR_20120322_1529_35.tar.gz" i.e. the file name contains slashes and double quotes!

tar cfz /this/file/does/nopt/exist .
tar: /this/file/does/nopt/exist: Cannot open: No such file or directory
tar: Error is not recoverable: exiting now

See the difference? There no double quotes around the file name/path in tar's error message.

It worked when you copy and paste the line because then, the doublequotes are intepreted by the shell.

Witness:

ls -l /tmp/screen-exchange
-rw-rw-rw- 1 aqn users 0 Mar 21 07:29 /tmp/screen-exchange
cmd='ls -l "'/tmp/screen-exchange'"'
$cmd
/bin/ls: "/tmp/screen-exchange": No such file or directory
eval $cmd
-rw-rw-rw- 1 aqn users 0 Mar 21 07:29 /tmp/screen-exchange

Of course, using eval won't guard against filenames with whitespaces in them. To guard against that, your tar command needs to be like so:

date>'file name with spaces'
file='file name with spaces'  # this is the equivalent of your $filename
cmd='ls -l "$file"'
$cmd
ls: "$file": No such file or directory
eval $cmd
-rw-r--r--  1 andyn  SPICE\Domain Users  1083 Mar 22 15:28 a b

Answer 3

If you execute the string $cmd, it won't work if "filename" embeds spaces You have to let bash creates the arguments. like this:

tar czvf "${filename}" $subdirs

You don't even need to put '\' in filename

Answer 4

Instead of mucking about with messy quoting issues you could get the results you want a different way and, perhaps, save some time. How about something like this?

#!/usr/bin/env bash

# abusing set -v for fun and profit

tar_output=/tmp/$$.tarout
tar_command=/tmp/$$.tarcmd
tmp_script=/tmp/$$.script

dir="$(cd "$(dirname "$0")"; pwd)"

cat>"${tmp_script}"<<-'END'
datestamp=$(date +%Y%m%d_%H%M_%S)
subdirs=(etc sbin bin)
backup_name=FOOBAR
filename="$1/Backup_${backup_name}_${date}.tar.gz"

printf 'tar cmd: '
set -v
tar czvf "$filename" "${subdirs[@]}" 2>"$2"
set +v
END

bash "${tmp_script}" "$dir" "${tar_output}" 2>"${tar_command}"
cat "${tar_command}" | head -n 1 | sed -e 's/2>"\$2"$//'
cat "${tar_output}"

rm -f "${tmp_script}" "${tar_command}" "${tar_output}"

I apologize for nothing, but in the real world note that you'd want to make proper temp files.

Answer 5

I would suggest you separate $cmd from $filename and $subdirs. I think the induced error comes from when you join these strings. Also, using multiple variables in one variable without proper quoting will also induce errors.

This should work for you:

cmd="tar -zcvf"
subdirs="etc/ sbin/ bin/"
filename="${DIR}/Backup_${backup_name}_${date}.tar.gz"
$cmd $filename $subdirs

Answer 6

#DIR is something like: /home/foo/foobar/test/ without any whitespace but can also include whitespace

DIR="$( cd "$( dirname "$0" )" && pwd )"
backup_name=FOOBAR
date=`date +%Y%m%d_%H%M_%S`
subdirs="etc/ sbin/ bin/"
filename="$DIR/Backup_$backup_name"_"$date.tar.gz"

cd /
echo "filename: $filename"
echo "subdirs $subdirs"
cmd="tar zcvf $filename $subdirs"
echo "cmd tar: $cmd"
$cmd