bashtestingtddautomated-testsextreme-programming

Reputation: 10493

Unit testing Bash scripts

We have a system that has some Bash scripts running besides Java code. Since we are trying to test everything that could possibly break, and those Bash scripts may break, we want to test them.

The problem is it is hard to test Bash scripts.

Is there a way or a best practice to test Bash scripts? Or should we quit using Bash scripts and look for alternative solutions that are testable?

Upvotes: 173

Answers (16)

Chemaclass

Reputation: 2011

I can highly recommend https://bashunit.typeddevs.com/

It is a pretty fresh and simple testing library for bash scripts.

Free and open source license
Easy to install
Well documented
Active community and engaged ownership

Upvotes: 3

cb2

Reputation: 711

Nikita Sobolev wrote an excellent blog post comparing a few different Bash test frameworks, Testing Bash applications. These are the frameworks listed in that post:

For the impatient: Nikita's conclusion was to use Bats, but it appears that Nikita missed the Bats-core project which appear to me to be the one to use going forward as the original Bats project has not been actively maintained since 2013.

Upvotes: 13

Leon S.

Reputation: 3687

I’ve tried a lot of the solutions presented here, but I found most of them too bulky and hard to use, so I built my own little testing framework: https://github.com/SnacOverflow/t-bash

It’s just one file in the repository that you can simply run directly, with a basic set of JUnit style asserts.

I’ve used it professionally in several internal projects and were able to make our Bash scripts super stable and regression resistant.

Upvotes: 2

Rolf

Reputation: 5743

Create a test mytest.sh. It calls your script with specific inputs.

Create a text file expected/mytest.stdout. It contains the expected output for the test.

Commit them to version control (if any).

Run the test and redirect the output to another file:

mytest.sh > actual/mytest.stdout

Then just us a text diff tool to see where the results deviate. I think you can just do

diff expected/mytest.stdout actual/mytest.stdout

Upvotes: 1

spbnick

Reputation: 5275

Epoxy is a Bash test framework I designed mainly for testing other software, but I use it to test Bash modules as well, including itself and Carton.

The main advantages are relatively low coding overhead, unlimited assertion nesting and flexible selection of assertions to verify.

I made a presentation comparing it to BeakerLib - a framework used by some at Red Hat.

Upvotes: 7

Koichi Nakashima

Reputation: 949

I created shellspec, because I wanted a easy-to-use and useful tool.

It written by a pure POSIX shell script. It has been tested with many shells more than shunit2. It has more powerful features than bats/bats-core.

For example, it supports nested block, easy to mock/stub, easy to skip/pending, parameterized tests, assertion line number, execute by line number, parallel execution, random execution, TAP/JUnit formatter, coverage and CI integration, profiler, etc.

See the demo on the project page.

Upvotes: 18

Daniel C. Sobral

Reputation: 297205

I can't believe no one talked about OSHT! It's compatible with both TAP and JUnit, it's pure shell (that is, no other languages involved), it works standalone too, and it's simple and direct.

Testing looks like this (snippets taken from the project page):

#!/bin/bash
. osht.sh

# Optionally, indicate number of tests to safeguard against abnormal exits
PLAN 13

# Comparing stuff
IS $(whoami) != root
var="foobar"
IS "$var" =~ foo
ISNT "$var" == foo

# test(1)-based tests
OK -f /etc/passwd
NOK -w /etc/passwd

# Running stuff
# Check exit code
RUNS true
NRUNS false

# Check stdio/stdout/stderr
RUNS echo -e 'foo\nbar\nbaz'
GREP bar
OGREP bar
NEGREP . # verify empty

# diff output
DIFF <<EOF
foo
bar
baz
EOF

# TODO and SKIP
TODO RUNS false
SKIP test $(uname -s) == Darwin

A simple run:

$ bash test.sh
1..13
ok 1 - IS $(whoami) != root
ok 2 - IS "$var" =~ foo
ok 3 - ISNT "$var" == foo
ok 4 - OK -f /etc/passwd
ok 5 - NOK -w /etc/passwd
ok 6 - RUNS true
ok 7 - NRUNS false
ok 8 - RUNS echo -e 'foo\nbar\nbaz'
ok 9 - GREP bar
ok 10 - OGREP bar
ok 11 - NEGREP . # verify empty
ok 12 - DIFF <<EOF
not ok 13 - TODO RUNS false # TODO Test Know to fail

The last test shows as "not ok", but the exit code is 0 because it's a TODO. One can set verbose as well:

$ OSHT_VERBOSE=1 bash test.sh # Or -v
1..13
# dcsobral \!= root
ok 1 - IS $(whoami) != root
# foobar =\~ foo
ok 2 - IS "$var" =~ foo
# \! foobar == foo
ok 3 - ISNT "$var" == foo
# test -f /etc/passwd
ok 4 - OK -f /etc/passwd
# test \! -w /etc/passwd
ok 5 - NOK -w /etc/passwd
# RUNNING: true
# STATUS: 0
# STDIO <<EOM
# EOM
ok 6 - RUNS true
# RUNNING: false
# STATUS: 1
# STDIO <<EOM
# EOM
ok 7 - NRUNS false
# RUNNING: echo -e foo\\nbar\\nbaz
# STATUS: 0
# STDIO <<EOM
# foo
# bar
# baz
# EOM
ok 8 - RUNS echo -e 'foo\nbar\nbaz'
# grep -q bar
ok 9 - GREP bar
# grep -q bar
ok 10 - OGREP bar
# \! grep -q .
ok 11 - NEGREP . # verify empty
ok 12 - DIFF <<EOF
# RUNNING: false
# STATUS: 1
# STDIO <<EOM
# EOM
not ok 13 - TODO RUNS false # TODO Test Know to fail

Rename it to use a .t extension and put it in a t subdirectory, and you can use prove(1) (part of Perl) to run it:

$ prove
t/test.t .. ok
All tests successful.
Files=1, Tests=13,  0 wallclock secs ( 0.03 usr  0.01 sys +  0.11 cusr  0.16 csys =  0.31 CPU)
Result: PASS

Set OSHT_JUNIT or pass -j to produce JUnit output. JUnit can also be combined with prove(1).

I have used this library both testing functions by sourcing their files and then running assertions with IS/OK and their negatives, and scripts by using RUN/NRUN. For me, this framework provides the most gain for the least overhead.

Upvotes: 8

Alexey Melezhik

Reputation: 971

Take a look at Outthentic. It is simple, extensible by many languages (Perl, Python, Ruby, and Bash on choice) and cross platform (Linux and Windows) framework to test any command line applications.

Upvotes: 0

Mark

Reputation: 6404

Give assert.sh a try:

source "./assert.sh"

local expected actual
expected="Hello"
actual="World!"
assert_eq "$expected" "$actual" "not equivalent!"
# => x Hello == World :: not equivalent!

Upvotes: 4

hrob

Reputation: 31

Maybe this can be used, or contributed to:

https://thorsteinssonh.github.io/bash_test_tools/

It is intended to write results in the TAP protocol which I imagine is good for CI and is good for those that want shell environments. I imagine some things run in shell environments, so some might argue it should be tested in their shell environment.

Upvotes: 3

pahaz

Reputation: 885

Try bashtest. It’s simple way to test your scripts. For example, you have do-some-work.sh which changes some configuration files. For example, add a new line, PASSWORD = 'XXXXX', to configuration file /etc/my.cfg.

You write Bash commands line by line and then check output.

Install:

pip3 install bashtest

Create tests is a just writing bash commands.

File test-do-some-work.bashtest:

# Run the script
$ ./do-some-work.sh > /dev/null

# Testing that the line "PASSWORD = 'XXXXX'" is in the file /etc/my.cfg
$ grep -Fxq "PASSWORD = 'XXXXX'" /etc/my.cfg && echo "YES"
YES

Run tests:

bashtest *.bashtest

You can find some examples here and here.

Upvotes: 3

Steve HHH

Reputation: 13147

I quite like shell2junit, a utility to generate JUnit-like output from Bash script tests. This is useful because the report generated can then be read by continuous integration systems, such as the JUnit plug-ins for Jenkins and Bamboo.

While shell2junit doesn't provide the comprehensive Bash scripting framework like shunit2, it does allow you have nice reporting of the test results.

Upvotes: 3

nimcap

Reputation: 10493

I got the following answer from a discussion group:

it's possible to import (include, whatever) a procedure (function, whatever it's named) from an external file. That's the key to writing a testing script: you break up your script into independent procedures that can then be imported into both your running script and your testing script, and then you have your running script be as simple as possible.

This method is like dependency injection for scripts and sounds reasonable. Avoiding Bash scripts and using more testable and less obscure language is preferable.

Upvotes: 33

Jim Dennis

Reputation: 17500

Why do you say that it's "hard" to test Bash scripts?

What's wrong with test wrappers like the following?

 #!/bin/bash
 set -e
 errors=0
 results=$($script_under_test $args<<ENDTSTDATA
 # inputs
 # go
 # here
 #
 ENDTSTDATA
 )
 [ "$?" -ne 0 ] || {
     echo "Test returned error code $?" 2>&1
     let errors+=1
     }

 echo "$results" | grep -q $expected1 || {
      echo "Test Failed.  Expected $expected1"
      let errors+=1
 }
 # And so on, et cetera, ad infinitum, ad nauseum
 [ "$errors" -gt 0 ] && {
      echo "There were $errors errors found"
      exit 1
 }

Upvotes: 6

ire_and_curses

Reputation: 70162

There is actually a shunit2, an xUnit based unit test framework for Bourne based shell scripts. I haven't used it myself, but it might be worth checking out.

Similar questions have been asked before:

Upvotes: 64

Janus Troelsen

Reputation: 21300

TAP-compliant Bash testing: Bash Automated Testing System

TAP, the Test Anything Protocol, is a simple text-based interface between testing modules in a test harness. TAP started life as part of the test harness for Perl but now has implementations in C, C++, Python, PHP, Perl, Java, JavaScript, and others.

bats-core

Upvotes: 42

Unit testing Bash scripts

Answers (16)

Related Questions