CODEWITHSUNDEEP
pythondjangomultithreadingheroku
zhuyxn
zhuyxn

Reputation: 7091

Using Threading to Update Database on Heroku?

creating a small web application to be deployed on Heroku, where we would like to update an entry in the database whenever a user clicks on an update button. Every day, we would automatically go through and update all the entries which had not been updated by some user in the last 24 hours.

The update function and the web application is written in Django, but because the update involves making a request to another website, it takes a few seconds to process, and I was thinking about making these requests perform in parallel, so multiple users can click on different update buttons at the same time, and have it go through quickly.

We were thinking about using the Scheduler on Heroku for the 24 hour update, but is there a threading application/implementation we could use for this kind of use case? Any tips would be awesome.

Upvotes: 2

Views: 620

Answers (1)

dokkaebi
dokkaebi

Reputation: 9190

Celery is great for scheduled tasks like that. I found a hack to set up workers on Heroku here, although I haven't used Heroku so you're on your own there.

The celery usage would look as simple as:

@celery.task
def do_costly_database_update(some_data):
    # do stuff
    pass

# And then in a view...
do_costly_database_update.delay(some_data)

Edit: Celery also handles recurring tasks like your nightly updates with a config entry like this:

'every-night': {
    'task': 'tasks.do_costly_database_update',
    'schedule': crontab(hour=0, minute=0),
    'args': some_data,
},

All of the Celery docs are really good.

Upvotes: 2

Related Questions