CODEWITHSUNDEEP
pythondjangoherokubackgroundcron
Martol1ni
Martol1ni

Reputation: 4702

Persistent background task Django + Heroku

I have a persistent background process currently running as a standalone python script on my ubuntu server, managed by supervisor. However, I am migrating to Heroku and wonder if anyone have any experiences setting up the same kind of environment.

The specifications of the script;

  1. Fetch information from external API
  2. Do calculations on the data
  3. Store the data to the database
  4. If the script used less than 5 seconds, sleep for the remaining time, else run again

I could run a cronjob every 5 seconds, but from time to time step 1-3 can take up to a full hour. Any tips?

Thanks.

Upvotes: 2

Views: 827

Answers (1)

rdegges
rdegges

Reputation: 33854

What you want to do is create a worker process. Simply define a command line script so that you can call it easily, then in your Procfile, add a new worker entry like so:

# Procfile
web: python manage.py runserver # example
worker: python manage.py start_cronjob # command to run your background process

Once you've got this defined in your Procfile, go ahead and push your app to Heroku, then scale up a worker process:

$ heroku scale worker=1

This will launch a single worker process.

To view the logs and ensure things are working as expected, you can say:

$ heroku logs --tail --ps worker

Upvotes: 3

Related Questions