CODEWITHSUNDEEP
pythondjangomultithreadingpython-daemon
AXG1010
AXG1010

Reputation: 1962

Long running background thread in Django model

I'm fairly new to Python and Django, so please let me know if there is a better way to do this. What I am trying to do is have each Device (which inherits from models.Model) kick off a long running background thread which constantly checks the health of that Device. However when I run my code, it does not seem to be executing like a daemon, as the server is sluggish and continually times out. This background thread will (in most cases) run the life of the program.

Below is a simplified version of my code:

class Device(models.Model):
    active = models.BooleanField(default=True)
    is_healthy = models.BooleanField(default=True)
    last_heartbeat = models.DateTimeField(null=True, blank=True)

    def __init__(self, *args, **kwargs):
        super(Device, self).__init__(*args, **kwargs)
        # start daemon thread that polls device's health
        thread = Thread(name='device_health_checker', target=self.health_checker())
        thread.daemon = True
        thread.start()


    def health_checker(self):
        while self.active:
            if self.last_heartbeat is not None:
                time_since_last_heartbeat = timezone.now() - self.last_heartbeat
                self.is_healthy = False if time_since_last_heartbeat.total_seconds() >= 60 else True
                self.save()
                time.sleep(10)

This seems like a very simple use of threading, but every time I search for solutions, the suggested approach is to use celery which seems like overkill to me. Is there a way to get this to work without the need for something like celery?

Upvotes: 2

Views: 2751

Answers (2)

AXG1010
AXG1010

Reputation: 1962

As @knbk mentioned in a comment, "Every time you query for devices, a new thread will be created for each device that is returned". This is something I originally overlooked.

However I was able to solve my issue using a single background thread that is kicked off as a Django application. This is a much simpler approach then adding a 3rd party library (like Celery).

class DeviceApp(AppConfig):
    name = 'device_app'

    def ready(self):
        # start daemon thread that polls device's health
        thread = Thread(name='device_health_checker', target=self.device_health_check)
        thread.daemon = True
        thread.start()

def device_health_check(self):
    while (true):
        for device in Device.objects.get_queryset():
            if device.last_heartbeat is not None:
                time_since_last_heartbeat = timezone.now() - device.last_heartbeat
                device.is_healthy = False if time_since_last_heartbeat.total_seconds() >= 60 else True
                device.save()
        time.sleep(10)

Upvotes: 1

Jonathan
Jonathan

Reputation: 8891

When you start out in your development environment the number of devices are likely quite low. So the number of threads perhaps are in the double digits as you test things out.

But this thread issue will rapidly become untenable as you increase the number of devices even if you got the code to work. So using celery with a celery beat is the better way to do it.

Also consider that you are new to Django and Python, trying to master threads on top of that would add even more complexity. Using celery for this would be a lot simpler and neater in the end.

Upvotes: 0

Related Questions