CODEWITHSUNDEEP
pythondjangoscheduled-tasks
Simeon Leyzerzon
Simeon Leyzerzon

Reputation: 19076

Job Scheduling in Django

I need to implement a scheduled task in our Django app. DBader's schedule seems to be a good candidate for the job, however when run it as part of a Django project, it doesn't seem to produce the desired effect.

Specifically, this works fine as an independent program:

import schedule
import time

import logging
log = logging.getLogger(__name__)

def handleAnnotationsWithoutRequests(settings):
    '''
    From settings passed in, grab job-ids list
    For each job-id in that list, perform annotation group/set logic [for details, refer to handleAnnotationsWithRequests(requests, username) 
                                                                     sans requests, those are obtained from db based on job-id ]
    '''
    print('Received settings: {}'.format(str(settings)))

def job():
    print("I'm working...")

#schedule.every(3).seconds.do(job)
#schedule.every(2).seconds.do(handleAnnotationsWithoutRequests, settings={'a': 'b'})
invoc_time = "10:33"
schedule.every().day.at(invoc_time).do(handleAnnotationsWithoutRequests, settings={'a': 'b'})

while True:
    schedule.run_pending()
    time.sleep(1)

But this (equivalent) code run in Django context doesn't result in an invocation.

def handleAnnotationsWithoutRequests(settings):
    '''
    From settings passed in, grab job-ids list
    For each job-id in that list, perform annotation group/set logic [for details, refer to handleAnnotationsWithRequests(requests, username) 
                                                                     sans requests, those are obtained from db based on job-id ]
    '''
    log.info('Received settings: {}'.format(str(settings)))

def doSchedule(settings):
    '''
    with scheduler library
    Based on time specified in settings, invoke .handleAnnotationsWithoutRequests(settings)
    '''
    #settings will need to be reconstituted from the DB first
    #settings = {}
    invocationTime = settings['running_at']
    import re
    invocationTime = re.sub(r'([AaPp][Mm])', "", invocationTime)
    log.info("Invocation time to be used: {}".format(invocationTime))
    schedule.every().day.at(invocationTime).do(handleAnnotationsWithoutRequests, settings=settings)

    while True:
        schedule.run_pending()
        time.sleep(1)

so the log from handleAnnotationsWithoutRequests() doesn't appear on the console.

Is this scheduling library compatible with Django? Are there any usage samples that one could refer me to?

I'm suspecting some thread issues are at work here. Perhaps there are better alternatives to be used? Suggestions are welcome.

Thank you in advance.

Upvotes: 1

Views: 10355

Answers (2)

Sam Texas
Sam Texas

Reputation: 1275

I do this a lot with Django Commands

The pattern I use is to setup a new Django command in my app and then make it a long-running process inside a never-ending while() loop.

I the loop iterates continuously with a custom defined sleep(1) timer.

The short version is here, with a bit of pseudo-code thrown in. You can see a working version of this pattern in my Django Reference Implementation.

class Command(BaseCommand):
    help = 'My Long running job'

    def handle(self, *args, **options):

        self.stdout.write(self.style.SUCCESS(f'Starting long-running job.'))

        while True:

            if conditions met for job:
                self.job()

            sleep(5)

    def job(self):
        self.stdout.write(self.style.SUCCESS(f'Running the job...'))

Upvotes: -1

AKX
AKX

Reputation: 168863

For web servers, you probably don't want something that runs in-process:

An in-process scheduler for periodic jobs [...]

https://github.com/Tivix/django-cron has proven a working solution.

There's also the heavyweight champion Celery and Celerybeat.

Upvotes: 2

Related Questions