CODEWITHSUNDEEP
djangolistviewpagination
omid jahadi
omid jahadi

Reputation: 161

django listview pagination is too slow on large dataset

I have a django listview that shows filtered data from Catalogue. It works fine on small database, but when database is too large, it takes too long time to return results.

views.py:

class all_ads(generic.ListView):
    paginate_by = 12
    template_name = 'new_list_view_grid-card.html'

    def get_queryset(self):
        city_district = self.request.GET.getlist('city_district')
        usage = self.request.GET.get('usage')
        status = self.request.GET.get('status')
        last2week = datetime.datetime.now() - datetime.timedelta(days=14)
        status = status.split(',')
        if usage:
            usage = usage.split(',')
        else:
            usage = ['1','2','3','4','5','6','7','8','9','10','11','12','13','14','15','16','17','18','19','20','21','22','23','24','25','26','27','28','29','30','31']
        intersections = list(set(status).intersection(usage))
        type_q = (Q(type__in=intersections) & Q(type__isnull=False))

        cat_ids = list(models.Catalogue.objects.filter(
                Q(*[Q(city__contains=x) for x in city_district], _connector=Q.OR) |
                Q(*[Q(district__contains=x) for x in city_district], _connector=Q.OR)
            ).values_list('pk', flat=True))

            result = models.Catalogue.objects.filter(
                Q(datetime__gte=last2week),
                type_q,
                pk__in=cat_ids
            ).distinct().order_by('-datetime').prefetch_related('type')

        return result

models.py:

class Catalogue(models.Model):
    city = models.CharField(db_index=True,max_length=100, null=True)
    district = models.CharField(db_index=True,max_length=100, null=True)
    type = models.ManyToManyField(Type, db_index=True)


class Type(models.Model):
    name = models.CharField(max_length=100, db_index=True)

    def __str__(self):
        return self.name

And this is template.html:

{% for Catalogue in catalogue_list %}
    "do something"
{% endfor %}

Upvotes: 0

Views: 330

Answers (1)

Mathias AG
Mathias AG

Reputation: 3

You can also decide to do it via the front-end, i.e. fetch all the data, send it to the front-end and display it with JavaScript

Upvotes: 0

Related Questions