CODEWITHSUNDEEP
linuxbashubuntucron
user3265370
user3265370

Reputation: 131

Running python program after it is terminated via crontab

I have a python webscraping program which needs to be scrapped continuously after the program is terminated. my technique is as follows

crontab -e (settings)

* * * * * /home/ahmed/Desktop/run.sh

run.sh

    TMP_FILE=/tmp/i_am_running
    [ -f $TMP_FILE ] && exit
    touch $TMP_FILE
    /usr/bin/python /home/ahmed/Desktop/python.py
    rm $TMP_FILE

The bash code must have some problem or may be my command in the crontab is wrong. the program is not running. Please guide

After Mark suggestions I modified the script like this

#!/bin/bash
PATH=$PATH:/bin:/usr/bin

date +'%H:%M:%S Started' >> /home/ahmed/Desktop/log.txt

TMP_FILE=/tmp/i_am_running
[ -f $TMP_FILE ] && exit
touch $TMP_FILE

date +'%H:%M:%S Starting Python' >> /home/ahmed/Desktop/log.txt
/usr/bin/python /home/ahmed/Desktop/python.py
rm $TMP_FILE

date +'%H:%M:%S Ended' >> /home/ahmed/Desktop/log.txt

The cron command i am using is * * * * * /home/ahmed/Desktop/run.sh

the log file which is created is this

15:21:01 Started
15:21:02 Starting Python
15:22:02 Started
15:23:01 Started
15:24:01 Started
15:24:30 Ended
15:25:01 Started
15:25:01 Starting Python
15:26:01 Started
15:27:18 Started
15:28:01 Started
15:29:01 Started
15:30:01 Started
15:31:01 Started
15:31:16 Ended
15:32:01 Started
15:32:01 Starting Python
15:33:01 Started
15:34:01 Started

It seems like the program is restarted before its ended. the log file should have starting program, started, ended, starting program, started, ended and so on.

Can someone guide me please?

Upvotes: 1

Views: 144

Answers (1)

Mark Setchell
Mark Setchell

Reputation: 208077

Have you made your script executable?

chmod +x /home/ahmed/Desktop/run.sh

Put a proper shebang and PATH in your script so it starts like this:

 #!/bin/bash
 PATH=$PATH:/bin:/usr/bin

Try your script on its own from the command line

/home/ahmed/Desktop/run.sh

If that doesn't work, change the shebang line to add -xv at the end

#!/bin/bash -xv

Check to see if /tmp/i_am_running exists

Check your cron log

grep CRON /var/log/syslog

Consider changing your script so you can see when it started and/or if it actually ran your python:

#!/bin/bash
PATH=$PATH:/bin:/usr/bin

date +'%H:%M:%S Started' >> /home/ahmed/Desktop/log.txt

TMP_FILE=/tmp/i_am_running
[ -f $TMP_FILE ] && exit
touch $TMP_FILE

date +'%H:%M:%S Starting Python' >> /home/ahmed/Desktop/log.txt
/usr/bin/python /home/ahmed/Desktop/python.py
rm $TMP_FILE

date +'%H:%M:%S Ended' >> /home/ahmed/Desktop/log.txt

By the way, I am not sure how running once at 18:01 constitutes "continuous scraping"?

Upvotes: 2

Related Questions