I have a shell script with this code: var=`hg st -R "$path"` if [ -n "$var" ]; then echo $var fi But the conditional code always executes, because hg st always prints at least one newline character. Is there a simple way to strip whitespace from $var (like trim() in PHP )? or Is there a standard way of dealing with this issue? I could use sed or AWK , but I'd like to think there is a more elegant solution to this problem.

Reputation: 91068

How to trim whitespace from a Bash variable?

I have a shell script with this code:

var=`hg st -R "$path"`
if [ -n "$var" ]; then
    echo $var
fi

But the conditional code always executes, because hg st always prints at least one newline character.

Is there a simple way to strip whitespace from $var (like trim() in PHP)?

Is there a standard way of dealing with this issue?

I could use sed or AWK, but I'd like to think there is a more elegant solution to this problem.

Upvotes: 1386

Answers (30)

pavd

Reputation: 173

Another solution using BASH-only built-ins:

This uses the ANSI-C quoting ($'...') and substring expansion (${var:0:-1}) features of BASH.

#!/bin/bash

string="$(cat /tmp/test.txt)"

while : ; do 
    case "$string" in
    $' '*|$'\n'*|$'\t'*) # trim leading spaces.
        string="${string:1}"
        ;;
    *$' '|*$'\n'|*$'\t') # trim trailing spaces.
        string="${string:0:-1}"
        ;;
    *)
        break
        ;;
    esac
done

printf '"%s"\n' "$string"

Content of /tmp/test.txt (an x surrounded with different types of white space):

The output:

"x"

But I actually prefer this solution, since it's POSIX-compatible and removes newlines as well (what I wanted). It's perhaps a bit faster too.

Upvotes: 0

Martín Zaragoza

Reputation: 1817

I know that the question wants to avoid using awk... however there is a very elegant one-liner to do it:

trim_path=$(echo "$path" | awk '$1=$1')

Upvotes: 1

bashfu

Reputation: 249

A solution that uses Bash built-ins called wildcards:

var="    abc    "
# remove leading whitespace characters
var="${var#"${var%%[![:space:]]*}"}"
# remove trailing whitespace characters
var="${var%"${var##*[![:space:]]}"}"   
printf '%s' "===$var==="

Here's the same wrapped in a function:

trim() {
    local var="$*"
    # remove leading whitespace characters
    var="${var#"${var%%[![:space:]]*}"}"
    # remove trailing whitespace characters
    var="${var%"${var##*[![:space:]]}"}"
    printf '%s' "$var"
}

You pass the string to be trimmed in quoted form, e.g.:

trim "   abc   "

This solution works with POSIX-compliant shells.

Reference

Remove leading & trailing whitespace from a Bash variable (original source)

Upvotes: 543

Takakiri

Reputation: 43

A simple answer is:

sed 's/^\s*\|\s*$//g'

An example:

$ before=$( echo -e " \t a  b \t ")
$ echo "(${before})"
(    a  b    )

$ after=$( echo "${before}"  |  sed 's/^\s*\|\s*$//g' )
$ echo "(${after})"
(a  b)

Upvotes: 9

IT professional

Reputation: 31

var = '  a b  '
# remove all white spaces
new=$(echo $var |  tr -d ' ')
# remove leading and trailing whitespaces
new=$(echo $var)

ab
a b

Upvotes: 3

Ivan

Reputation: 7307

Create an array instead of variable this will trim all space, tab and newline characters:

arr=( $(hg st -R "$path") )
if [[ -n "${arr[@]}" ]]; then
    printf -- '%s\n' "${arr[@]}"
fi

Upvotes: 0

Adam Erickson

Reputation: 6363

There are a few different options purely in BASH:

line=${line##+([[:space:]])}    # strip leading whitespace;  no quote expansion!
line=${line%%+([[:space:]])}   # strip trailing whitespace; no quote expansion!
line=${line//[[:space:]]/}   # strip all whitespace
line=${line//[[:space:]]/}   # strip all whitespace

line=${line//[[:blank:]]/}   # strip all blank space

The former two require extglob be set/enabled a priori:

shopt -s extglob  # bash only

NOTE: variable expansion inside quotation marks breaks the top two examples!

The pattern matching behaviour of POSIX bracket expressions are detailed here. If you are using a more modern/hackable shell such as Fish, there are built-in functions for string trimming.

Upvotes: 13

Ondra Žižka

Reputation: 46904

The simplest way for the single-line use cases I know of:

echo "  ABC  " | sed -e 's# \+\(.\+\) \+#\1#'

How it works:

-e enables advanced regex
I use # with sed as I don't like the "messy library" patterns like /\////\/\\\/\/
sed wants most regex control chars escaped, hence all the \
Otherwise it's just ^ +(.+) +$, i.e. spaces at the beginning, a group no.1, and spaces at the end.
All this is replaced with just "group no.1".

Therefore, ABC becomes ABC.

This should be supported on most recent systems with sed.

For tabs, that would be

echo "  ABC  " | sed -e 's#[\t ]\+\(.\+\)[\t ]\+#\1#'

For multi-line content, that already needs character classes like [:space:] as described in other answers, and may not be supported by all sed implementations.

Reference: Sed manual

Upvotes: 0

Roger Keays

Reputation: 3247

read already trims whitespace, so in bash you can do this:

$ read foo <<< "   foo  bar   two spaces follow   "
$ echo ".$foo."
.foo  bar   two spaces follow.

The POSIX compliant version is a bit longer

$ read foo << END
   foo  bar   two spaces follow   
END
$ echo ".$foo."
.foo  bar   two spaces follow.

Upvotes: 1

Eric Yriarte

Reputation: 51

The simplest and cheapest way to do this is to take advantage of echo ignoring spaces. So, just use

dest=$(echo $source)

for instance:

> VAR="   Hello    World   "
> echo "x${VAR}x"
x   Hello    World   x
> TRIMD=$(echo $VAR)
> echo "x${TRIMD}x"
xHello Worldx

Note that this also collapses multiple whitespaces into a single one.

Upvotes: 3

rkachach

Reputation: 17375

In order to remove all the spaces from the beginning and the end of a string (including end of line characters):

echo $variable | xargs echo -n

This will remove duplicate spaces also:

echo "  this string has a lot       of spaces " | xargs echo -n

Produces: 'this string has a lot of spaces'

Upvotes: 130

gMale

Reputation: 17895

If you have shopt -s extglob enabled, then the following is a neat solution.

This worked for me:

text="   trim my edges    "

trimmed=$text
trimmed=${trimmed##+( )} #Remove longest matching series of spaces from the front
trimmed=${trimmed%%+( )} #Remove longest matching series of spaces from the back

echo "<$trimmed>" #Adding angle braces just to make it easier to confirm that all spaces are removed

#Result
<trim my edges>

To put that on fewer lines for the same result:

text="    trim my edges    "
trimmed=${${text##+( )}%%+( )}

Upvotes: 14

Mario Palumbo

Reputation: 995

The "trim" function removes all horizontal whitespace:

ltrim () {
    if [[ $# -eq 0 ]]; then cat; else printf -- '%s\n' "$@"; fi | perl -pe 's/^\h+//g'
    return $?
}

rtrim () {
    if [[ $# -eq 0 ]]; then cat; else printf -- '%s\n' "$@"; fi | perl -pe 's/\h+$//g'
    return $?
}

trim () {
    ltrim "$@" | rtrim
    return $?
}

Upvotes: 0

David Farrell

Reputation: 457

Array assignment expands its parameter splitting on the Internal Field Separator (space/tab/newline by default).

words=($var)
var="${words[@]}"

Upvotes: 2

Moses Davidowitz

Reputation: 982

This is what I did and worked out perfect and so simple:

the_string="        test"
the_string=`echo $the_string`
echo "$the_string"

Output:

test

Upvotes: 17

Daniel Alder

Reputation: 5382

There are a lot of answers, but I still believe my just-written script is worth being mentioned because:

it was successfully tested in the shells bash/dash/busybox shell
it is extremely small
it doesn't depend on external commands and doesn't need to fork (->fast and low resource usage)
it works as expected:
- it strips all spaces and tabs from beginning and end, but not more
- important: it doesn't remove anything from the middle of the string (many other answers do), even newlines will remain
- special: the "$*" joins multiple arguments using one space. if you want to trim & output only the first argument, use "$1" instead
- if doesn't have any problems with matching file name patterns etc

The script:

trim() {
  local s2 s="$*"
  until s2="${s#[[:space:]]}"; [ "$s2" = "$s" ]; do s="$s2"; done
  until s2="${s%[[:space:]]}"; [ "$s2" = "$s" ]; do s="$s2"; done
  echo "$s"
}

Usage:

mystring="   here     is
    something    "
mystring=$(trim "$mystring")
echo ">$mystring<"

Output:

>here     is
    something<

Upvotes: 26

makevoid

Reputation: 3297

A simple answer is:

echo "   lol  " | xargs

Xargs will do the trimming for you. It's one command/program, no parameters, returns the trimmed string, easy as that!

Note: this doesn't remove all internal spaces so "foo bar" stays the same; it does NOT become "foobar". However, multiple spaces will be condensed to single spaces, so "foo bar" will become "foo bar". In addition it doesn't remove end of lines characters.

Upvotes: 1630

Olivier Meurice

Reputation: 562

I had to test the result (numeric) from a command but it seemed the variable with the result was containing spaces and some non printable characters. Therefore even after a "trim" the comparison was erroneous. I solved it by extracting the numerical part from the variable:

numerical_var=$(echo ${var_with_result_from_command} | grep -o "[0-9]*")

Upvotes: 4

GuruM

Reputation: 885

From Bash Guide section on globbing

To use an extglob in a parameter expansion

 #Turn on extended globbing  
shopt -s extglob  
 #Trim leading and trailing whitespace from a variable  
x=${x##+([[:space:]])}; x=${x%%+([[:space:]])}  
 #Turn off extended globbing  
shopt -u extglob

Here's the same functionality wrapped in a function (NOTE: Need to quote input string passed to function):

trim() {
    # Determine if 'extglob' is currently on.
    local extglobWasOff=1
    shopt extglob >/dev/null && extglobWasOff=0 
    (( extglobWasOff )) && shopt -s extglob # Turn 'extglob' on, if currently turned off.
    # Trim leading and trailing whitespace
    local var=$1
    var=${var##+([[:space:]])}
    var=${var%%+([[:space:]])}
    (( extglobWasOff )) && shopt -u extglob # If 'extglob' was off before, turn it back off.
    echo -n "$var"  # Output trimmed string.
}

Usage:

string="   abc def ghi  ";
#need to quote input-string to preserve internal white-space if any
trimmed=$(trim "$string");  
echo "$trimmed";

If we alter the function to execute in a subshell, we don't have to worry about examining the current shell option for extglob, we can just set it without affecting the current shell. This simplifies the function tremendously. I also update the positional parameters "in place" so I don't even need a local variable

trim() {
    shopt -s extglob
    set -- "${1##+([[:space:]])}"
    printf "%s" "${1%%+([[:space:]])}" 
}

so:

$ s=$'\t\n \r\tfoo  '
$ shopt -u extglob
$ shopt extglob
extglob         off
$ printf ">%q<\n" "$s" "$(trim "$s")"
>$'\t\n \r\tfoo  '<
>foo<
$ shopt extglob
extglob         off

Upvotes: 53

NOYB

Reputation: 695

# Strip leading and trailing white space (new line inclusive).
trim(){
    [[ "$1" =~ [^[:space:]](.*[^[:space:]])? ]]
    printf "%s" "$BASH_REMATCH"
}

# Strip leading white space (new line inclusive).
ltrim(){
    [[ "$1" =~ [^[:space:]].* ]]
    printf "%s" "$BASH_REMATCH"
}

# Strip trailing white space (new line inclusive).
rtrim(){
    [[ "$1" =~ .*[^[:space:]] ]]
    printf "%s" "$BASH_REMATCH"
}

# Strip leading and trailing white space (new line inclusive).
trim(){
    printf "%s" "$(rtrim "$(ltrim "$1")")"
}

# Strip leading and trailing specified characters.  ex: str=$(trim "$str" $'\n a')
trim(){
    if [ "$2" ]; then
        trim_chrs="$2"
    else
        trim_chrs="[:space:]"
    fi

    [[ "$1" =~ ^["$trim_chrs"]*(.*[^"$trim_chrs"])["$trim_chrs"]*$ ]]
    printf "%s" "${BASH_REMATCH[1]}"
}

# Strip leading specified characters.  ex: str=$(ltrim "$str" $'\n a')
ltrim(){
    if [ "$2" ]; then
        trim_chrs="$2"
    else
        trim_chrs="[:space:]"
    fi

    [[ "$1" =~ ^["$trim_chrs"]*(.*[^"$trim_chrs"]) ]]
    printf "%s" "${BASH_REMATCH[1]}"
}

# Strip trailing specified characters.  ex: str=$(rtrim "$str" $'\n a')
rtrim(){
    if [ "$2" ]; then
        trim_chrs="$2"
    else
        trim_chrs="[:space:]"
    fi

    [[ "$1" =~ ^(.*[^"$trim_chrs"])["$trim_chrs"]*$ ]]
    printf "%s" "${BASH_REMATCH[1]}"
}

# Strip leading and trailing specified characters.  ex: str=$(trim "$str" $'\n a')
trim(){
    printf "%s" "$(rtrim "$(ltrim "$1" "$2")" "$2")"
}

Building upon moskit's expr soulution...

# Strip leading and trailing white space (new line inclusive).
trim(){
    printf "%s" "`expr "$1" : "^[[:space:]]*\(.*[^[:space:]]\)[[:space:]]*$"`"
}

# Strip leading white space (new line inclusive).
ltrim(){
    printf "%s" "`expr "$1" : "^[[:space:]]*\(.*[^[:space:]]\)"`"
}

# Strip trailing white space (new line inclusive).
rtrim(){
    printf "%s" "`expr "$1" : "^\(.*[^[:space:]]\)[[:space:]]*$"`"
}

# Strip leading and trailing white space (new line inclusive).
trim(){
    printf "%s" "$(rtrim "$(ltrim "$1")")"
}

Upvotes: 13

Brian Cain

Reputation: 57

Strip one leading and one trailing space

trim()
{
    local trimmed="$1"

    # Strip leading space.
    trimmed="${trimmed## }"
    # Strip trailing space.
    trimmed="${trimmed%% }"

    echo "$trimmed"
}

For example:

test1="$(trim " one leading")"
test2="$(trim "one trailing ")"
test3="$(trim " one leading and one trailing ")"
echo "'$test1', '$test2', '$test3'"

Output:

'one leading', 'one trailing', 'one leading and one trailing'

Strip all leading and trailing spaces

trim()
{
    local trimmed="$1"

    # Strip leading spaces.
    while [[ $trimmed == ' '* ]]; do
       trimmed="${trimmed## }"
    done
    # Strip trailing spaces.
    while [[ $trimmed == *' ' ]]; do
        trimmed="${trimmed%% }"
    done

    echo "$trimmed"
}

For example:

test4="$(trim "  two leading")"
test5="$(trim "two trailing  ")"
test6="$(trim "  two leading and two trailing  ")"
echo "'$test4', '$test5', '$test6'"

Output:

'two leading', 'two trailing', 'two leading and two trailing'

Upvotes: 81

MattyV

Reputation: 845

Let's define a variable containing leading, trailing, and intermediate whitespace:

FOO=' test test test '
echo -e "FOO='${FOO}'"
# > FOO=' test test test '
echo -e "length(FOO)==${#FOO}"
# > length(FOO)==16

How to remove all whitespace (denoted by [:space:] in tr):

FOO=' test test test '
FOO_NO_WHITESPACE="$(echo -e "${FOO}" | tr -d '[:space:]')"
echo -e "FOO_NO_WHITESPACE='${FOO_NO_WHITESPACE}'"
# > FOO_NO_WHITESPACE='testtesttest'
echo -e "length(FOO_NO_WHITESPACE)==${#FOO_NO_WHITESPACE}"
# > length(FOO_NO_WHITESPACE)==12

How to remove leading whitespace only:

FOO=' test test test '
FOO_NO_LEAD_SPACE="$(echo -e "${FOO}" | sed -e 's/^[[:space:]]*//')"
echo -e "FOO_NO_LEAD_SPACE='${FOO_NO_LEAD_SPACE}'"
# > FOO_NO_LEAD_SPACE='test test test '
echo -e "length(FOO_NO_LEAD_SPACE)==${#FOO_NO_LEAD_SPACE}"
# > length(FOO_NO_LEAD_SPACE)==15

How to remove trailing whitespace only:

FOO=' test test test '
FOO_NO_TRAIL_SPACE="$(echo -e "${FOO}" | sed -e 's/[[:space:]]*$//')"
echo -e "FOO_NO_TRAIL_SPACE='${FOO_NO_TRAIL_SPACE}'"
# > FOO_NO_TRAIL_SPACE=' test test test'
echo -e "length(FOO_NO_TRAIL_SPACE)==${#FOO_NO_TRAIL_SPACE}"
# > length(FOO_NO_TRAIL_SPACE)==15

How to remove both leading and trailing spaces--chain the seds:

FOO=' test test test '
FOO_NO_EXTERNAL_SPACE="$(echo -e "${FOO}" | sed -e 's/^[[:space:]]*//' -e 's/[[:space:]]*$//')"
echo -e "FOO_NO_EXTERNAL_SPACE='${FOO_NO_EXTERNAL_SPACE}'"
# > FOO_NO_EXTERNAL_SPACE='test test test'
echo -e "length(FOO_NO_EXTERNAL_SPACE)==${#FOO_NO_EXTERNAL_SPACE}"
# > length(FOO_NO_EXTERNAL_SPACE)==14

Alternatively, if your bash supports it, you can replace echo -e "${FOO}" | sed ... with sed ... <<<${FOO}, like so (for trailing whitespace):

FOO_NO_TRAIL_SPACE="$(sed -e 's/[[:space:]]*$//' <<<${FOO})"

Upvotes: 1219

Alois Mahdal

Reputation: 11253

Use:

trim() {
    local orig="$1"
    local trmd=""
    while true;
    do
        trmd="${orig#[[:space:]]}"
        trmd="${trmd%[[:space:]]}"
        test "$trmd" = "$orig" && break
        orig="$trmd"
    done
    printf -- '%s\n' "$trmd"
}

It works on all kinds of whitespace, including newline,
It does not need to modify shopt.
It preserves inside whitespace, including newline.

Unit test (for manual review):

#!/bin/bash

. trim.sh

enum() {
    echo "   a b c"
    echo "a b c   "
    echo "  a b c "
    echo " a b c  "
    echo " a  b c  "
    echo " a  b  c  "
    echo " a      b  c  "
    echo "     a      b  c  "
    echo "     a  b  c  "
    echo " a  b  c      "
    echo " a  b  c      "
    echo " a N b  c  "
    echo "N a N b  c  "
    echo " Na  b  c  "
    echo " a  b  c N "
    echo " a  b  c  N"
}

xcheck() {
    local testln result
    while IFS='' read testln;
    do
        testln=$(tr N '\n' <<<"$testln")
        echo ": ~~~~~~~~~~~~~~~~~~~~~~~~~ :" >&2
        result="$(trim "$testln")"
        echo "testln='$testln'" >&2
        echo "result='$result'" >&2
    done
}

enum | xcheck

Upvotes: 4

moskit

Reputation: 94

Use:

var=`expr "$var" : "^\ *\(.*[^ ]\)\ *$"`

It removes leading and trailing spaces and is the most basic solution, I believe. Not Bash built-in, but 'expr' is a part of coreutils, so at least no standalone utilities are needed like sed or AWK.

Upvotes: 2

VAmp

Reputation: 1

You can trim simply with echo:

foo=" qsdqsd qsdqs q qs   "

# Not trimmed
echo \'$foo\'

# Trim
foo=`echo $foo`

# Trimmed
echo \'$foo\'

Upvotes: 56

David Newcomb

Reputation: 10943

While it's not strictly Bash this will do what you want and more:

php -r '$x = trim("  hi there  "); echo $x;'

If you want to make it lowercase too, do:

php -r '$x = trim("  Hi There  "); $x = strtolower($x) ; echo $x;'

Upvotes: -3

Luca Borrione

Reputation: 17022

I would simply use sed:

function trim
{
    echo "$1" | sed -n '1h;1!H;${;g;s/^[ \t]*//g;s/[ \t]*$//g;p;}'
}

a) Example of usage on single-line string

string='    wordA wordB  wordC   wordD    '
trimmed=$( trim "$string" )

echo "GIVEN STRING: |$string|"
echo "TRIMMED STRING: |$trimmed|"

Output:

GIVEN STRING: |    wordA wordB  wordC   wordD    |
TRIMMED STRING: |wordA wordB  wordC   wordD|

b) Example of usage on multi-line string

string='    wordA
   >wordB<
wordC    '
trimmed=$( trim "$string" )

echo -e "GIVEN STRING: |$string|\n"
echo "TRIMMED STRING: |$trimmed|"

Output:

GIVEN STRING: |    wordAA
   >wordB<
wordC    |

TRIMMED STRING: |wordAA
   >wordB<
wordC|

c) Final note:
If you don't like to use a function, for single-line string you can simply use a "easier to remember" command like:

echo "$string" | sed -e 's/^[ \t]*//' | sed -e 's/[ \t]*$//'

Example:

echo "   wordA wordB wordC   " | sed -e 's/^[ \t]*//' | sed -e 's/[ \t]*$//'

Output:

wordA wordB wordC

Using the above on multi-line strings will work as well, but please note that it will cut any trailing/leading internal multiple space as well, as GuruM noticed in the comments

string='    wordAA
    >four spaces before<
 >one space before<    '
echo "$string" | sed -e 's/^[ \t]*//' | sed -e 's/[ \t]*$//'

Output:

wordAA
>four spaces before<
>one space before<

So if you do mind to keep those spaces, please use the function at the beginning of my answer!

d) EXPLANATION of the sed syntax "find and replace" on multi-line strings used inside the function trim:

sed -n '
# If the first line, copy the pattern to the hold buffer
1h
# If not the first line, then append the pattern to the hold buffer
1!H
# If the last line then ...
$ {
    # Copy from the hold to the pattern buffer
    g
    # Do the search and replace
    s/^[ \t]*//g
    s/[ \t]*$//g
    # print
    p
}'

Upvotes: 8

Avenger

Reputation: 21

trim() removes whitespaces (and tabs, non-printable characters; I am considering just whitespaces for simplicity). My version of a solution:

var="$(hg st -R "$path")" # I often like to enclose shell output in double quotes
var="$(echo "${var}" | sed "s/\(^ *\| *\$\)//g")" # This is my suggestion
if [ -n "$var" ]; then
 echo "[${var}]"
fi

The 'sed' command trims only leading and trailing whitespaces, but it can be piped to the first command as well resulting in:

var="$(hg st -R "$path" | sed "s/\(^ *\| *\$\)//g")"
if [ -n "$var" ]; then
 echo "[${var}]"
fi

Upvotes: 4

Gilles Quénot

Reputation: 185620

Use this simple Bash parameter expansion:

$ x=" a z     e r ty "
$ echo "START[${x// /}]END"
START[azerty]END

Upvotes: 3

TrinitronX

Reputation: 5233

I needed to trim whitespace from a script when the IFS variable was set to something else. Relying on Perl did the trick:

# trim() { echo $1; } # This doesn't seem to work, as it's affected by IFS

trim() { echo "$1" | perl -p -e 's/^\s+|\s+$//g'; }

strings="after --> , <-- before,  <-- both -->  "

OLD_IFS=$IFS
IFS=","
for str in ${strings}; do
  str=$(trim "${str}")
  echo "str= '${str}'"
done
IFS=$OLD_IFS

Upvotes: 4

How to trim whitespace from a Bash variable?

Answers (30)

Reference

Strip one leading and one trailing space

Strip all leading and trailing spaces

Related Questions