CODEWITHSUNDEEP
linuxdockerscheduled-taskscron
user1566200
user1566200

Reputation: 1838

Solution for running scheduled cron job within a Docker container?

I have a Docker container in which I have my Python tools installed, including my Luigi pipeline interface. I would like to run a shell script which kicks off my Luigi pipeline on a weekly basis using cron.

I have tried high and low to get cron to work within a Docker container. I cannot, for the life of me, get my crontab -e file to run.

In my file I have:

0 0 * * Sun /data/myscript.sh

followed by a new line. Cron is running in the background - ps aux | grep cron shows /usr/sbin/cron is running. Furthermore, in my /var/log/syslog file, I have:

/USR/SBIN/CRON[2037]: (root) CMD (/data/myscript.sh)

I've also tried using 0 0 * * Sun . /root/.bashrc ; sh /data/myscript.sh

However, my script does not run (when I run my script manually using bash myscript.sh, I get the expected results).

Suggestions?

Upvotes: 4

Views: 16264

Answers (2)

Jrgns
Jrgns

Reputation: 25188

A container is meant to only one run main process. You either need to run crond as the main process for a container, or ensure that crond is running alongside your main process. This kind of breaks the contracts / point of containers, but sometimes it's easier to set it up this way. Instructions below:

My Dockerfile has the following ENTYPOINT:

ENTRYPOINT ["/startup.sh"]

And then within startup.sh I do a couple of things to spin up the container, but most importantly before executing the last command, I have this:

crond
exec start_my_service

crond starts the daemon that executes the crons, and start_my_service then becomes the primary process for my container.

Upvotes: 4

Dockstar
Dockstar

Reputation: 1028

Scheduled tasks won't run inside of a normal container since there is no scheduler running. The only active task will be that which you have elected to run via the CMD keyword or the Entrypoint.

In order to execute schedule tasks, it's more prudent to utilize the host scheduler and docker exec commands:

docker exec <container> <command>
docker exec <container> /data/myscript.sh

So you would end up with a cron on your host something like :

(Crontab Style) 0 * * * * root docker exec mycontainer /data/myscript.sh

If you have a cluster, you would have to query the cluster first to locate the container, or even have a script do it for you.

Upvotes: 10

Related Questions