CODEWITHSUNDEEP
bashshellcron
Alex
Alex

Reputation: 25

Different results when running a script by cron than manually executed

I have a script that makes some curl calls to a list of destinations found in list.txt file using keys found in keys.txt, each call result is saved in a different txt file that starts with the server name +_show_counters. When I run this script manually it runs correctly and give me the right responses from the right servers found in list.txt but when ran by cron the information provided does not correspond to the right server but to another which is 4 positions later in the list.

this is the code:

#!/bin/bash
PATH=/usr/local/bin:/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/sbin:/opt/puppetlabs/bin
temp=/local/scripts/shell/checker/temp/
timestamp=$(date +'%Y_%m_%d--%H:%M:%S')

paste $temp"list.txt" $temp"keys.txt" > $temp"combined.txt"
awk '{ printf "https://%s//api/?type=op&cmd=blablabla&key=%s\n", $1, $2}' $temp"combined.txt" > $temp"allCurls.txt"

count=1
for crl in `more +1 ${temp}allCurls.txt`
do
        filename=`awk -v count=$count 'NR==count{print $1}'  ${temp}list.txt`
        filekey=${filename}
        filename="$temp${filename}_show_counters.txt"
        curl -k $crl > ${filename}
        echo curl -k   $crl  >> $temp"curls_debug.txt"
        count=$((count+1))
done

This is the cron content:

0,10,20,30,40,50 * * * * /local/scripts/shell/checker/checker.sh > /dev/null 2>&1

I already tried editing the cron to start with below commands but does not make difference.

SHELL=/bin/bash
PATH=/usr/local/bin:/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/sbin:/opt/puppetlabs/bin

I found in /var/spool/mail/ below errors, then all consecutive curls right executed which is then causing the wrong order in the results provided.

curl: (6) Couldn't resolve host '::::::::::::::'
curl: (3) <url> malformed
curl: (6) Couldn't resolve host '::::::::::::::'

So I added the echo curls_debug.txt to the for and found:

curl -k ::::::::::::::
curl -k /local/scripts/shell/checker/temp/allCurls.txt
curl -k ::::::::::::::

This only happens when executed by cron, If I run this script manually then the curls_debug.txt shows the curls in the expected format composed of curl -k https://server_name+command+key since the very first curl and therefore brings right results.

Can any one of you please give me a hand to see what is wrong here and how to fix it?, thanks a lot.

Upvotes: 2

Views: 515

Answers (1)

Gordon Davisson
Gordon Davisson

Reputation: 125798

This is a guess, but I suspect it's due to using more to read the file -- more is an interactive tool, and shouldn't be used in non-interactive scripts. Using cat would be better, but a while read ... done <file loop is generally the best way to do this. And rather than using a messy set of intermediate files, I'd just use a single combined while read loop to read both files in parallel (via file descriptors #3 and #4, just to keep things clean), and put the info from the two files together directly:

#!/bin/bash
PATH=/usr/local/bin:/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/sbin:/opt/puppetlabs/bin
temp=/local/scripts/shell/checker/temp
timestamp=$(date +'%Y_%m_%d--%H:%M:%S')

while read domain <&3 && read key <&4; do
    crl="https://${domain}//api/?type=op&cmd=blablabla&key=${key}"
    filename="${temp}/${domain}_show_counters.txt"
    curl -k "$crl" > "${filename}"
done 3<"${temp}/list.txt" 4<"${temp}/keys.txt"

Note: I also cleaned up the quoting (in general, variable references should be in double-quotes), and left the trailing / off of the temp variable (I find it much clearer to add it when using the variable -- makes it clear we're referring to a file in that directory).

Upvotes: 2

Related Questions