CODEWITHSUNDEEP
linuxbashcp
Mr.Zhou
Mr.Zhou

Reputation: 143

Why the Linux command CP behave differently in CLI and in script?

I want to copy a bunch of Verilog/systemverilog sources, so I use CP with wildcard expression:

cp <some_dir>/*.{v,sv,svh} .

It works. But when I put it to a script with exactly the same line, the CP command fails with the log:

cp: cannot stat `../../mytest/spiTest/*.{v,sv,svh}': No such file or directory

How is that happening?

PS: I use bash as the shell.

And here is my script:

#!/bin/bash
rdir=../../mytest/spiTest
f1="$rdir/bench.lst"
f2="$rdir/cphex" #the script to copy rom data
f3="$rdir/make*" #makefile scripts
f4="$rdir/*.hex" #rom files
f5="$rdir/*.{v,sv,svh}" #testbench files
echo 'Copying files...'
cp $f1 $f2 $f3 $f4 .
cp $f5 .

I do changed the first line to 

#!/bin/bash -vx

and run this script again, and I get:

#!/bin/bash -vx

rdir=../../mytest/spiTest
+ rdir=../../mytest/spiTest
f1="$rdir/bench.lst"
+ f1=../../mytest/spiTest/bench.lst
f2="$rdir/cphex" #the script to copy rom data
+ f2=../../mytest/spiTest/cphex
f3="$rdir/make*" #makefile scripts
+ f3='../../mytest/spiTest/make*'
f4="$rdir/*.hex" #rom files
+ f4='../../mytest/spiTest/*.hex'
f5="$rdir/*.{v,sv,svh}" #testbench files
+ f5='../../mytest/spiTest/*.{v,sv,svh}'

echo 'Copying files...'
+ echo 'Copying files...'
Copying files...
cp $f1 $f2 $f3 $f4 .
+ cp ../../mytest/spiTest/bench.lst ../../mytest/spiTest/cphex ../../mytest/spiTest/makefile ../../mytest/spiTest/makefile.defines ../../mytest/spiTest/rom.hex ../../mytest/spiTest/rom_if.hex .
cp $f5 .
+ cp '../../mytest/spiTest/*.{v,sv,svh}' .
cp: cannot stat `../../mytest/spiTest/*.{v,sv,svh}': No such file or directory

Upvotes: 4

Views: 1437

Answers (3)

glenn jackman
glenn jackman

Reputation: 246799

Here's the problem: the order of substitutions. Bash performs brace expansion before variable expansion. In the line cp $f5 ., bash will do:

  1. brace expansion: n/a
    • this is the key point: the variable contains a brace expression, but the shell does not see it now when it needs to.
  2. tilde expansion: n/a
  3. parameter expansion: yes -- cp ../../mytest/spiTest/*.{v,sv,svh} .
  4. command substitution: n/a
  5. arithmetic expansion: n/a
  6. process substitution: n/a
  7. word splitting: n/a
  8. filename expansion: yes, bash looks for files in that directory ending with the string
    .{v,sv,svh}. It finds none, nullglob is not set, thus the pattern is not removed from the command
  9. quote removal: n/a

Now the command is executed and fails with the error you see.

https://www.gnu.org/software/bash/manual/bashref.html#Shell-Expansions

Solutions:

  1. use Aaron's idea of an array
  2. (not recommended) force a 2nd round of expansions: eval cp $f5 .

Upvotes: 4

Basile Starynkevitch
Basile Starynkevitch

Reputation: 1

The line

 f5="$rdir/*.{v,sv,svh}" #testbench files

is probably wrong. First, avoid comments at end of line, they should be (at least for readability) in a separate line. Then, avoid using globbing in variable assignment. So remove that line, and code later (that is, replace the old cp $f5 . line with)

 cp "$rdir"/*.{v,sv,svh} .

BTW, I would test that "$rdir" is indeed a directory with

 if [ ! -d "$rdir" ] ; then
    echo invalid directory $rdir > /dev/stderr
    exit 1
 fi

You should read the Advanced Bash Scripting Guide

Upvotes: 1

Aaron Digulla
Aaron Digulla

Reputation: 328584

Check the first line of the script. It probably reads:

#!/bin/sh

which switches the shell from BASH to Bourne Shell. Use

#!/bin/bash

instead.

[EDIT] You're running into problems with expansion. BASH has a certain order in which it expands patterns and variables. That means:

f5="$rdir/*.{v,sv,svh}" #testbench files

is quoted, so no file name expansion happens at this time. Only the variable $rdir is expanded. When

cp $f5 .

is executed, BASH first looks for file names to expand and there are none. Then it expands variables (f5) and then calls cp with two arguments: ../../mytest/spiTest/*.{v,sv,svh} and .. Since cp expects the shell to have performed the file name expansion already, you get an error.

To fix this, you have to use arrays:

f5=($rdir/*.{v,sv,svh})

This replaces the variable and then expands the file names and puts everything into the array f5. You can then call cp with this array while preserving whitespaces:

cp "${f5[@]}" .

Every single character here is important. [@] tells BASH to expand the whole array here. The quotes say: Preserve whitespace. {} is necessary to tell BASH that [@] is part of the variable "name" to expand.

Upvotes: 5

Related Questions