CODEWITHSUNDEEP
pythondjangodjango-models
Andrea
Andrea

Reputation: 603

Django 1.10 Delete large cascade querySet

I'm trying to delete all objects from a large queryset. Here is my models.py

from __future__ import unicode_literals
from django.contrib.auth.models import User
from django.db import models


class Fund(models.Model):

    name = models.CharField(max_length=255, blank=False, null=False)
    start_date = models.DateField(default=None, blank=False, null=False)

    def __unicode__(self):
        return self.name


class FundData(models.Model):

    fund = models.ForeignKey(Fund, on_delete=models.CASCADE)
    date = models.DateField(default=None, blank=False, null=False)
    value = models.FloatField(default=None, blank=True, null=True)

    def __unicode__(self):
        return "{} --- Data: {} --- Value: {} ".format(str(self.fund), str(self.date), str(self.value))

But when I try to delete all records the query take too much time and mysql is being timed out.

Fund.objects.all().delete()

Upvotes: 1

Views: 2072

Answers (2)

Andrea
Andrea

Reputation: 603

Finally I got a simple solution to delete all cascade object breaking down the macro deletion operation preferring delete Fund objects one by one. Because it is a really long operation (approximately 1 second for each object for thousands objects) I assigned it to a celery worker. Slow but safe I think, if someone got a better solution please let me know!

@shared_task
def reset_funds():
    for fund in Fund.objects.all():
        print "Delete Fund: {}".format(fund.name)
        fund.delete()
    return "All Funds Deleted!"

Upvotes: 1

pythad
pythad

Reputation: 4267

First of all you can change the timeout time of MySQL by edditing settings.py

DATABASES = {
    'default': {
        ...
        OPTIONS = {
            'connect_timeout': 5, # your timeout time
        }
        ...
    }
}

The reasons why .delete() may be slow are:

  1. Django has to ensure cascade deleting functions properly. It has to look for foreign keys to your models
  2. Django has to somehow handle pre and post-save signals for your models

If you are sure that your models don't have cascade deleting or any signals to be handled, you can try to use private _raw_delete as follows:

queryset._raw_delete(queryset.db)

You can find more details on it here

Upvotes: 1

Related Questions