Maths in a while loop causing random negative numbers

Question

So I have done this in both python and bash, and the code I am about to post probably has a world of things wrong with it but it is generally very basic and I cannot see a reason that it would cause this 'bug' which I will explain soon.. I have done the same in Python, but much more professionally and cleanly and it also causes this error (at some point, the maths generates a negative number, which makes no sense.)

#!/bin/bash
while [ 1 ];
do

zero=0
ARRAY=()
ARRAY2=()

first=`command to generate a list of numbers`
sleep 1
second=`command to generate a list of numbers`

# so now we have two data sets, 1 second between the capture of each.

for i in $first;
do
ARRAY+=($i)
done

for i in $second;
do
ARRAY2+=($i)
done

for (( c=$zero; c<=${#ARRAY2[@]}; c++ ))
do
expr ${ARRAY2[$c]} - ${ARRAY[$c]}
done

ARRAY=()
ARRAY2=()
zero=0
c=0
first=``
second=``
math=''

done

So the script grabs a set of data, waits 1 second, grabs it again, does math on the two sets to get the difference, that difference is printed. It's very simple, and I have done it elegantly in Python too - no matter how I would do it every now and then, could be anywhere from 3 loops in to 30 loops in, we will get negative numbers.. like so:

START 0 0 0 0 0 19 10 563 0 -34 19 14 2 0 -1302 1198 -532 639 -1078 1119 1 0 0 -843 33 880 0 5 -8 -13508 8773 4541 988 181 -12 -205 217 -9 7 1 -360 303 60 1 0 0 -12 -96 98 3 -870 904 -130 -2105 2264 6 -3084 1576 1650 -939 971 -2249 1150 1281

-693 9 513 142 76 expr: syntax error

Please help, I simply can't find anything about this.

Sample OUTPUT as requested:

ARRAY1 OUTPUT
1 15 1 25 25 1 2 1 3541 853 94567 42 5 1 351 51 1 11 1 13 7 14 12 3999 983 5 1938 3 8287 40 1 1 1 5253 706 1 1 1 1 5717 3 50 1 85 100376 17334 4655 1 1345 2 1 16 1777 1 3 38 23 8 32 47 781 947 1 1 206 9 1 3 2 81 2602 7 158 1 1 43 91 1 120 6589 6 2534 1092 1 6014 7 2 2 37 1 1 1 80 2 1 1270 15448 66 1 10238 1 10794 16061 4 1 1 1 9754 5617 1123 926 3 24 10 16
ARRAY2 OUTPUT
1 15 1 25 25 1 2 1 3555 859 95043 42 5 1 355 55 1 11 1 13 7 14 12 4015 987 5 1938 3 8335 40 1 1 1 5280 706 1 1 1 1 5733 3 50 1 85 100877 17396 4691 1 1353 2 1 16 1782 1 3 38 23 8 32 47 787 947 1 1 206 9 1 3 2 81 2602 7 159 1 1 43 91 1 120 6869 6 2534 1092 1 6044 7 2 2 37 1 1 1 80 2 1 1270 15563 66 1 10293 1 10804 16134 4 1 1 1 9755 5633 1135 928 3 24 10 16
START

Answer 1

I think the problem has to be when the two arrays don't have the same size. It's easy to reproduce that syntax error -- one of the operands for the minus operator is an empty string:

$ a=5; b=3; expr $a - $b
2
$ a=""; b=3; expr $a - $b
expr: syntax error
$ a=5; b=""; expr $a - $b
expr: syntax error
$ a=""; b=""; expr $a - $b
-

Try

ARRAY=( $(command to generate a list of numbers) )
sleep 1
ARRAY2=( $(command to generate a list of numbers) )

if (( ${#ARRAY[@]} != ${#ARRAY2[@]} )); then
    echo "error: different size arrays!"
    echo "ARRAY: ${#ARRAY[@]} (${ARRAY[*]})"
    echo "ARRAY2: ${#ARRAY2[@]} (${ARRAY2[*]})"
fi

---

"The error occurs whenever the first array is smaller than the second" -- of course. You're looping from 0 to the array size of ARRAY2. When ARRAY has fewer elements, you'll eventually try to access an index that does not exist in the array. When you try to reference an unset variable, bash gives you the empty string.

$ a=(1 2 3)
$ b=(4 5 6 7)
$ i=2; expr ${a[i]} - ${b[i]}
-3
$ i=3; expr ${a[i]} - ${b[i]}
expr: syntax error

Answer 2

The answer lies in Russell Uhl's comment above. Your loop runs one time to many(this is your code):

for (( c=$zero; c<=${#ARRAY2[@]}; c++ ))
do
        expr ${ARRAY2[$c]} - ${ARRAY[$c]}
done

To fix, you need to change the test condition from c <= ${#ARRAY2[@]} to c < ${#ARRAY2[@]}:

for (( c=$zero; c < ${#ARRAY2[@]}; c++ ))
do
        echo $((${ARRAY2[$c]} - ${ARRAY[$c]}))
done

I've also changed the expr to use arithmetic evaluation builtin $((...)).

The test script (sum.sh):

#!/bin/bash
zero=0
ARRAY=()
ARRAY2=()

first="1 15 1 25 25 1 2 1 3541 853 94567 42 5 1 351 51 1 11 1 13 7 14 12 3999 983 5 1938 3 8287 40 1 1 1 5253 706 1 1 1 1 5717 3 50 1 85 100376 17334 4655 1 1345 2 1 16 1777 1 3 38 23 8 32 47 7
second="1 15 1 25 25 1 2 1 3555 859 95043 42 5 1 355 55 1 11 1 13 7 14 12 4015 987 5 1938 3 8335 40 1 1 1 5280 706 1 1 1 1 5733 3 50 1 85 100877 17396 4691 1 1353 2 1 16 1782 1 3 38 23 8 32 47

for i in $first; do
    ARRAY+=($i)
done

# Alternately as chepner suggested:
ARRAY2=($second)

for (( c=$zero; c < ${#ARRAY2[@]}; c++ )); do
    echo -n $((${ARRAY2[$c]} - ${ARRAY[$c]})) " "
done

Running it:

samveen@precise:/tmp$ echo $BASH_VERSION
4.2.25(1)-release
samveen@precise:/tmp$ bash sum.sh
0  0  0  0  0  0  0  0  14  6  476  0  0  0  4  4  0  0  0  0  0  0  0  16  4  0  0  0  48  0  0  0  0  27  0  0  0  0  0  16  0  0  0  0  501  62  36  0  8  0  0  0  5  0  0  0  0  0  0  0  6  0  0  0  0  0  0  0  0  0  0  0  1  0  0  0  0  0  0  280  0  0  0  0  30  0  0  0  0  0  0  0  0  0  0  0  115  0  0  55  0  10  73  0  0  0  0  1  16  12  2  0  0  0  0

EDIT:
* Added improvements from suggestions in comments.