CODEWITHSUNDEEP
bashubuntucron
studentneedshelp
studentneedshelp

Reputation: 53

Properly formating a crontab executable on a bash script

I’m having some serious issues with trying to get the proper format for my bash script to be able to run successfully in crontab. The bash script runs successfully when manually prompted from the command line. Here is the bash script in question (the actual parameters themselves [$1 & $2] have been manually placed in the script):

#!/bin/bash
# Usage: ./s3DeleteByDateVirginia "bucketname" "file type"

past=$(date +"%F" -d "60 days ago")

aws s3api list-objects --bucket $1 --query 'Contents[?LastModified<=`'$past'`][].{Key:Key}' | grep $2 | while read -r line
do
fileName=`echo $line`
aws s3api delete-object  --bucket $1 --key "$fileName"
done;

The script is in this bash file: /home/ubuntu/s3DeleteByDateVirginiaSoco1 To set up the script I use: sudo crontab –e Now I see people online saying you need to give it the proper path which doesn’t make any sense to me especially when it comes to putting it in the right location because I’m seeing a number of various modifications of this online but it consists of this format: SHELL=/bin/sh sPATH=/bin:/sbin:/usr/bin:/usr/sbin but I don't know where to put it.

According to the syslog the cron functionalities that parts working but the script itself doesn’t execute:syslog

In addition to this the script has all of the proper permissions to run. All in all, I’m more confused that when I started and I’m not seeing that much documentation on how crontab works.

Crontab in question: enter image description here

Additional Edits based on user's suggestions: Here's my polished script: enter image description here

Here's the crontab line: 

 # m h  dom mon dow   command

PATH=/usr/local/bin:/usr/bin:/bin:/root/.local/bin/aws
33 20 * * *  /home/ubuntu/s3DeleteByDateSoco1

Updated syslog: enter image description here

Upvotes: 0

Views: 1290

Answers (1)

Gordon Davisson
Gordon Davisson

Reputation: 125788

Ok, I see several problems here. First, you need to put this in the crontab file for the user you want the script to run as. If you want to run it under your user account, do not use just crontab -e instead of sudo crontab -e (with sudo, it edits the root user's crontab file).

Second, you need to use the correct path & name for the script; it looks like it's /home/ubuntu/s3DeleteByDateVirginiaSoco1, so that's what should be in the crontab entry. Don't add ".sh" if it's not actually part of the filename. It also looks like you tried adding "root" in front of the path; don't do that either, since crontab will try to execute "root" as a command, and it'll fail. bash -c doesn't hurt, but it doesn't help at all either, so don't use it.

Third, the PATH needs to be set appropriately for the executables you use in the script. By default, cron jobs execute with a PATH of just "/usr/bin:/bin", so when you use a command like aws, it'll look for it as /usr/bin/aws, not find it, look for it as /usr/aws, not find it, and give the error "aws: command not found" that you see in the last log entry. First, you need to find out where aws (and any other programs your script depends on) are; you can use which aws in your regular shell to find this out. Suppose it's /usr/local/bin/aws. Then you can either:

  • Add a line like PATH=/usr/local/bin:/usr/bin:/bin (with maybe any other directories you think are appropriate) to the crontab file, before the line that says to run your script.

  • Add a line like PATH=/usr/local/bin:/usr/bin:/bin (with maybe any other directories you think are appropriate) to the your script file, before the lines that use aws.

  • In your script, use an explicit path every time you want to run aws (something like /usr/local/bin/aws s3api list-objects ...)

You can use any (or all) of the above, but you must use at least one or it won't be able to find the aws command (or anything else that isn't in the set of core commands that come with the OS).

Fourth, I don't see where $1 and $2 are supplied. You say they've been manually placed in the script, but I don't know what you mean by that. Since the script expects them as parameters, you need to specify them in the crontab file (i.e. the command in crontab should be something like /home/ubuntu/s3DeleteByDateVirginiaSoco1 bucketname pattern).

Fifth, the script itself doesn't follow good quoting conventions. In general, all variable references should be in double-quotes. For example, use grep "$2" instead of grep $2. Without the double-quotes, variables that contain spaces or certain shell metacharacters can cause weird parsing problems.

Finally, why do you do fileName=echo $line (with backquotes I can't replicate here)? This mostly just copies the value of $line into the variable fileName, but can have those weird parsing problems I mentioned in the last point. If you want to copy a variable reliably, just use fileName="$line" (or fileName=$line -- this is one of the few cases where it's safe to leave the double-quotes off).

BTW, shellcheck.net is good at spotting common problems like bad quoting; I recommend running your scripts through it to see what it finds.

Upvotes: 2

Related Questions