Script runs when executed but fails when sourced

Question

Original Title: Indirect parameter substitution breaks when the script is sourced (zsh)

zsh 5.7.1 (x86_64-apple-darwin19.0)

GNU bash, version 4.4.20(1)-release (x86_64-pc-linux-gnu)

I’m developing a shell script on a Mac and I’m trying to keep it portable between bash & zsh, so array indexing is a consideration. I know that I can set KSH_ARRAYS to get indexing to start at 0, but I decided to query the OS for the shell that’s in use and set the start index accordingly, which led to the issue described below.

It made sense (to me anyway!) to use indirect expansion, which is what led to the problem. Consider the script indirect.sh:

#! /bin/bash

declare -r ARRAY_START_BASH=0
declare -r ARRAY_START_ZSH=1

declare -r SHELL_BASH=0
declare -r SHELL_ZSH=1

# Indirect expansion is used to reference the values of the variables declared
# in this case statement e.g. ${!ARRAY_START}
case $(basename $SHELL) in
  "bash" )
    declare -r SHELL_ID=SHELL_BASH
    declare -r ARRAY_START=ARRAY_START_BASH
    ;;

  "zsh" )
    declare -r SHELL_ID=SHELL_ZSH
    declare -r ARRAY_START=ARRAY_START_ZSH
    ;;

  * )
    return 1
    ;;

esac

echo "Shell ID: ${!SHELL_ID} Index arrays from: ${!ARRAY_START}"

It works fine when run from the command line while in the same directory:

<my home> ~ % echo "$(./indirect.sh)"
Shell ID: 1 Index arrays from: 1

Problems arise when I source the script:

<my home> ~ % echo "$(. ~/indirect.sh)"
/Users/<me>/indirect.sh:28: bad substitution

I don’t understand why sourcing the script changes the behavior of the parameter expansion.

Is this expected behavior? If so, I’d be grateful if someone could explain it and hopefully, offer a work around.

Answer 1

To add to your answer (what I'm going to say is too long for a comment), I can not think of any application, why your script could be useful if not sourced. Actually, I came accross the need of such a script by myself in exactly one occasion:

Since I use as interactive shell not only zsh, but also sometimes bash, so I have written my .zshrc and .bashrc to set up everything (including defining variables and shell functions for interactive use). In order to safe work, I try to put code which works under both bash and zsh into a single file (say: .commonrc), and my .zshrc and .bashrc have inside them a

source .commonrc

While many things are so different in bash and zsh, that I can't put them into .commonrc, some can, provided I do some tweaking. One reason for headache is obviously the different indexing of arrays, which you seemingly try to solve. So I have also a similar feature. However, I don't nee ca case construct for this. Instead, my .bashrc looks like this (using your naming of the variables):

...
declare -r ARRAY_START=0
source .commonrc
...

and my .zshrc looks like this:

...
declare -r ARRAY_START=1
source .commonrc
...

Since it does not happen that the .bashrc is run from a zsh and vice versa, I don't need to query what kind of shell I have.

Answer 2

The problem described in the original post has nothing to do with indirect expansion. The difference in behavior is a result of different shells being invoked depending on whether the script is “executed” or “sourced”. These differences reveal the basic flaw in deriving the shell from the $SHELL variable that underpins the script's design. If the shell defined in $SHELL does not match the shebang, the script will fail either when sourced or executed. An explanation follows.

Indirect expansion doesn’t offer value in the given scenario because values could just as easily be assigned directly. They’ll have to be assigned that way regardless given the different syntax used for indirect expansion between shells. In fact, other syntax differences between shells makes the entire premise for detecting the shell moot! However, putting that aside, the difference in behavior is a result of different shells being invoked based on whether the script is “executed” or “sourced”. The behavior of sourcing is well documented with numerous explanations on the web, but for context here’s how it works:

Executing a Script Use the “./“ syntax to execute a script. When run this way, the script executes in a sub-shell. Any changes the script makes to it’s shell are applied to the sub-shell, not the shell in which the script was launched, so those changes are lost when the shell exits because the sub-shell in which it executed is destroyed as well. For example, if the script changes the working directory, it does so in the sub-shell. The working directory of the main shell that launched the script is unchanged when the script terminates. If you want to make changes to the shell in which the script was launched, it must be sourced.

Sourcing a Script Use the “source “ syntax to source a script. When run this way, the script essentially becomes an argument for the source command, which handles invoking the appropriate execution. Some shells (e.g. ksh) use a single period “.” instead of “source”.

When a script is executed with the “./“ syntax, the shebang at the top of the file is used to determine which shell to use. When a script is sourced, the shebang is ignored and the shell in which the script is launched is used instead. Also note that the period that appears in the “./“ command syntax used to execute a script, is not related to the period that’s occasionally used as an alias for the source command.

The script in the post uses bash in the shebang statement, so it works when executed because it’s run using bash. When it’s sourced from zsh, it encounters the incorrect indirect expansion syntax:

“${!A_VAR}"

The correct syntax is:

"${(P)A_VAR}"

However, correcting the syntax won’t help because it will then fail when executed. The shebang will invoke bash and the syntax will be wrong again. That renders indirection useless for accessing a variable designed to indicate the shell in use. More importantly, a design based on querying an environment variable for the shell is flawed due to differences in the shell that’s ultimately used depending on whether the script is executed or sourced.