Is there a general way to add/prepend/remove paths from general environment variables in bash?

Question

I am wondering if anyone has a good way to add/remove paths from environment variables in bash (not just the PATH variable).

My situation is that I am implementing a simple module-like system on a cluster where I do not have admin privileges. I want to be able to write scripts like:

openmpi_module.sh

#!/bin/bash

if [ $1 == "load" ]; then
  path_prepend PATH "$HOME/apps/openmpi-1.8.1-gcc-4.9.0/bin"
  path_prepend CPATH "$HOME/apps/openmpi-1.8.1-gcc-4.9.0/include"
  ...
fi

if [ $1 == "unload" ]; then
  path_remove PATH "$HOME/apps/openmpi-1.8.1-gcc-4.9.0/bin"
  path_remove CPATH "$HOME/apps/openmpi-1.8.1-gcc-4.9.0/include"
  ...
fi

I have found questions dealing with modifying the PATH variable specifically, but not a general environment variable. It seems like a general version of those solutions would be a useful tool to have.

My current solution is to source the following in my .bash_profile, but I am wondering if anyone has a more elegant solution (short of installing a local copy of the actual Modules system). Mostly I feel uncomfortable with the use of so many evals and echos, a practice I prefer to avoid.

#!/bin/bash
# modified from https://stackoverflow.com/questions/370047/

function path_remove () {
eval export $(echo $1=\$\(echo -n $(echo -n "\$$1 | awk -v RS=: -v ORS=: '\$0 != \"'$2'\"' | sed 's/:\$//'")\))
}

function path_append () {
path_remove $1 $2
eval export $1="\$$1:$2"
}

function path_prepend () {
path_remove $1 $2
eval export $1="$2:\$$1"
}

Answer 1

Here are compact bash functions, which:

• avoid eval by using printf -v for setting a variable indirectly, as in @Charles Duffy's excellent answer.
• use just parameter expansion to manipulate the path list.
• are self-contained (they do not rely on each other - at the expense of duplicating some code).
• force pre/appending, as in the OP's functions: i.e., if the specified path entry already exists but in a different position, it is forced into the desired position.

Caveat: Existing duplicate entries are only removed if they're not directly adjacent (see below for an alternative).

#!/usr/bin/env bash

# The functions below operate on PATH-like variables whose fields are separated
# with ':'.
# Note: The *name* of the PATH-style variable must be passed in as the 1st
#       argument and that variable's value is modified *directly*.

# SYNOPSIS: path_prepend varName path
# Note: Forces path into the first position, if already present.
#       Duplicates are removed too, unless they're directly adjacent.
# EXAMPLE: path_prepend PATH /usr/local/bin
path_prepend() {
  local aux=":${!1}:"
  aux=${aux//:$2:/:}; aux=${aux#:}; aux=${aux%:}
  printf -v "$1" '%s' "${2}${aux:+:}${aux}"  
}

# SYNOPSIS: path_append varName path
# Note: Forces path into the last position, if already present.
#       Duplicates are removed too, unless they're directly adjacent.
# EXAMPLE: path_append PATH /usr/local/bin
path_append() {
  local aux=":${!1}:"
  aux=${aux//:$2:/:}; aux=${aux#:}; aux=${aux%:}
  printf -v "$1" '%s' "${aux}${aux:+:}${2}"
}

# SYNOPSIS: path_remove varName path
# Note: Duplicates are removed too, unless they're directly adjacent.
# EXAMPLE: path_remove PATH /usr/local/bin
path_remove() {
  local aux=":${!1}:"
  aux=${aux//:$2:/:}; aux=${aux#:}; aux=${aux%:}
  printf -v "$1" '%s' "$aux"
}

---

If you do need to deal with directly adjacent duplicates and/or want to be able to specify a different field separator, here are more elaborate functions that use the array technique from @konsolebox' helpful answer.

# SYNOPSIS: field_prepend varName fieldVal [sep]
#   SEP defaults to ':'
# Note: Forces fieldVal into the first position, if already present.
#       Duplicates are removed, too.
# EXAMPLE: field_prepend PATH /usr/local/bin
field_prepend() {
    local varName=$1 fieldVal=$2 IFS=${3:-':'} auxArr
    read -ra auxArr <<< "${!varName}"
    for i in "${!auxArr[@]}"; do
        [[ ${auxArr[i]} == "$fieldVal" ]] && unset auxArr[i]
    done
    auxArr=("$fieldVal" "${auxArr[@]}")
    printf -v "$varName" '%s' "${auxArr[*]}"
}

# SYNOPSIS: field_append varName fieldVal [sep]
#   SEP defaults to ':'
# Note: Forces fieldVal into the last position, if already present.
#       Duplicates are removed, too.
# EXAMPLE: field_append PATH /usr/local/bin
field_append() {
    local varName=$1 fieldVal=$2 IFS=${3:-':'} auxArr
    read -ra auxArr <<< "${!varName}"
    for i in "${!auxArr[@]}"; do
        [[ ${auxArr[i]} == "$fieldVal" ]] && unset auxArr[i]
    done
    auxArr+=("$fieldVal")
    printf -v "$varName" '%s' "${auxArr[*]}"
}

# SYNOPSIS: field_remove varName fieldVal [sep]
#   SEP defaults to ':'
# Note: Duplicates are removed, too.
# EXAMPLE: field_remove PATH /usr/local/bin
field_remove() {
    local varName=$1 fieldVal=$2 IFS=${3:-':'} auxArr
    read -ra auxArr <<< "${!varName}"
    for i in "${!auxArr[@]}"; do
        [[ ${auxArr[i]} == "$fieldVal" ]] && unset auxArr[i]
    done
    printf -v "$varName" '%s' "${auxArr[*]}"
}

Answer 2

The easiest way and probably most consistent is to split those strings first into arrays:

IFS=: read -ra T <<< "$PATH"

Then add elements to those arrays:

# Append

T+=("$SOMETHING")

# Prepend or Insert

T=("$SOMETHING" "${T[@]}")

# Remove

for I in "${!T[@]}"; do
    if [[ ${T[I]} == "$SOMETHING" ]]; then
        unset "T[$I]"
        break  ## You can skip breaking if you want to remove all matches not just the first one.
    fi
done

After that you can put it back with a safe eval:

IFS=: eval 'PATH="${T[*]}"'

Actually if you're a bit "conservative", you can save IFS first:

OLD_IFS=$IFS; IFS=:
PATH="${T[*]}"
IFS=$OLD_IFS

Functions:

shopt -s extglob

function path_append {
    local VAR=$1 ELEM=$2 T
    IFS=: read -ra T <<< "${!VAR}"
    T+=("$ELEM")
    [[ $VAR == [[:alpha:]_]*([[:alnum:]_]) ]] && IFS=: eval "$VAR=\${T[*]}"
}

function path_prepend {
    local VAR=$1 ELEM=$2 T
    IFS=: read -ra T <<< "${!VAR}"
    T=("$ELEM" "${T[@]}")
    [[ $VAR == [[:alpha:]_]*([[:alnum:]_]) ]] && IFS=: eval "$VAR=\${T[*]}"
}

function path_remove {
    local VAR=$1 ELEM=$2 T
    IFS=: read -ra T <<< "${!VAR}"
    for I in "${!T[@]}"; do
        [[ ${T[I]} == "$ELEM" ]] && unset "T[$I]"
    done
    [[ $VAR == [[:alpha:]_]*([[:alnum:]_]) ]] && IFS=: eval "$VAR=\${T[*]}"
}

Answer 3

export sets a flag specifying that a variable should be in the environment. If it's already there, though, updates will always be passed through to the environment; you don't need to do anything else.

Thus:

PATH=/new/value:$PATH

or

PATH=$PATH:/new/value

...is entirely sufficient, unless you want to add your own logic (as around deduplication).

If you want to act only if no duplicate values exist, you might write something like the following:

prepend() {
  local var=$1
  local val=$2
  local sep=${3:-":"}
  [[ ${!var} =~ (^|"$sep")"$val"($|"$sep") ]] && return # already present
  [[ ${!var} ]] || { printf -v "$var" '%s' "$val" && return; } # empty
  printf -v "$var" '%s%s%s' "$val" "$sep" "${!var}" # prepend
}

append() {
  local var=$1
  local val=$2
  local sep=${3:-":"}
  [[ ${!var} =~ (^|"$sep")"$val"($|"$sep") ]] && return # already present
  [[ ${!var} ]] || { printf -v "$var" '%s' "$val" && return; } # empty
  printf -v "$var" '%s%s%s' "${!var}" "$sep" "${val}" # append
}

remove() {
  local var=$1
  local val=$2
  local sep=${3:-":"}
  while [[ ${!var} =~ (^|.*"$sep")"$val"($|"$sep".*) ]]; do
    if [[ ${BASH_REMATCH[1]} && ${BASH_REMATCH[2]} ]]; then
      # match is between both leading and trailing content
      printf -v "$var" '%s%s' "${BASH_REMATCH[1]%$sep}" "${BASH_REMATCH[2]}"
    elif [[ ${BASH_REMATCH[1]} ]]; then
      # match is at the end
      printf -v "$var" "${BASH_REMATCH[1]%$sep}"
    else
      # match is at the beginning
      printf -v "$var" "${BASH_REMATCH[2]#$sep}"
    fi
  done
}

...used as:

prepend PATH /usr/local/bin
remove PATH /usr/local/bin

Note that:

• There's no need for the function keyword, which breaks POSIX compatibility for no good reason. (Other things we do break POSIX, but actually add value in some way).
• Our regular expression quotes things which should be interpreted literally, and leaves things which should be treated as regex characters unquoted. Note that this behavior is only entirely consistent in versions of bash after 3.2; if you need compatibility with older releases, some updates are called for.