CODEWITHSUNDEEP
linuxbashshellglob
user9685224
user9685224

Reputation:

Why do commands get expanded by Bash before they run?

I was running the command find . -name *.bak and was receiving errors "Paths must precede expression" errors. This question has been answered and the answer is quite good, but what I don't understand is why does bash expand the wild card before the find command is ran? original example.

It says find . -name *.bak is expanded to find . -name tim.bak example.bak. Can someone explain why it is expanded, I can't understand why it would be a good idea to have this feature for globs.

Upvotes: 1

Views: 263

Answers (3)

Gordon Davisson
Gordon Davisson

Reputation: 125838

This is a design decision that was made very early in unix history. Filename wildcards need to be expanded at some point, but there's a choice about whether it should be done by the command interpreter (aka shell), and the result (list of matching files) handed to the executable, or whether the command interpreter should just pass what it's given to the executable, and have it do the expansion. Different OSes do this differently; unix does the first, but VMS (which I used before migrating to unix) does it the second way. Both have advantages and disadvantages.

  • The unix way's primary advantage is that the wildcard expansion code only has to be written and used in one place: in the shell. Commands just plain don't have to worry about it. Second, you can improve/expand the matching syntax (e.g. bash's extglob syntax) by making a change in one place. Third, you get consistent expansion syntax between all the different commands (vs. having to learn different rules for different commands, like the basic vs. extended vs. perl-compatible regex mess).

  • The VMS way's primary advantage is that the executable program knows what the arguments mean, and can change/suppress expansion appropriately. For example, find would know not to expand your wildcards in the current directory, grep would know not to try to expand the regex pattern as though it was a file wildcard, scp could expand a wildcard on the remote computer, etc. The second advantage is that wildcards can be used in ways that the unix system simply doesn't allow, because the program has more insight into how the arguments are specified; for example rename *.jpeg *.jpg is (if I remember right) a perfectly good VMS command that does exactly what it looks like it should do.

    [Edit] Another advantage is that it avoids the risk of filenames being mistaken for command options. This can be a serious security problem with the unix approach, since anyone who can control filenames may also be able to control commands & scripts that work with those files. For example, creating a file named "-e somecommand" would cause rsync -t * foo:src/ to execute somecommand on the remote computer. This article gives more examples.

    The need for each program to do its own wildcard expansion isn't nearly as bad as you'd think. There are standard library functions that handle expansion, so all a program needs to do is call that and then work on the resulting file list, no big deal. And those library functions can be expanded just like shell syntax can be, and standardizing on them provides consistency across programs, etc.

As you might be able to tell from the above, I overall think the VMS way is better. It's a little more work for the programmers, but has significant benefits in usability and power. But I'm sure that's a minority view among unix people, and anyway it'd take way a massive effort to ever change how unix does it, so as long is unix is unix, it's not going to change.

Upvotes: 6

cdarke
cdarke

Reputation: 44364

One of UNIX's founding fathers, Dennis Ritchie, gave this explanation:

"Putting this expansion mechanism into the shell has several advantages: the code only appears once, so no space is wasted and commands in general need take no special action; the algorithm is certain to be applied uniformly."

D.M.Ritchie, in Unix Time-Sharing System: A Retrospective BSTJ July-Aug 1978

Upvotes: 0

Allan
Allan

Reputation: 12438

A very brief answer:

It is the way Bash works, it does always expand what he can, replace the variable by their values, use the environment variables (as $PATH for example), interpret the aliases and pass the result (the corresponding arguments) to the "proper" command (extracted from $PATH) you are calling. The argument expansion can be seen as one of bash numerous responsibilities.

Imagine, if you didn't have this feature, you would need to implement for each program you develop this feature of expanding the arguments properly (with the corresponding files in the working directory or in the directory you are pointing to). In brief it would be a nightmare!!! That's why it is centralized in bash.

Last but not least, for your find command, change it that way to avoid any interference from bash. By doing this, the find command will receive literally the regex for the filenames to search and manage it properly 

find . -name '*.bak'
             ^     ^

List of Shell responsibilities:

http://www.informit.com/articles/article.aspx?p=31480&seqNum=4 http://ptgmedia.pearsoncmg.com/images/chap3_0672324903/elementLinks/03fig08.jpg http://tldp.org/LDP/abs/html/x9644.html

Upvotes: 1

Related Questions