CODEWITHSUNDEEP
pythondjangocountadmin
Simon Steinberger
Simon Steinberger

Reputation: 6825

How to override queryset count() method in Django's admin list

In order to avoid time consuming and costly exact database count queries, I'd like to override the count() method inside a Django admin class like so:

from django.contrib import admin
from django.db import connection

class CountProxy:
    def __call__(self):
        # how to access the queryset `query` here?
        query = ...

        try:
            if not query.where:
                cursor = connection.cursor()
                cursor.execute("SELECT reltuples FROM pg_class WHERE relname = %s", [query.model._meta.db_table])
                n = int(cursor.fetchone()[0])
                if n >= 1000: return n # exact count for small tables
            return object_list.count()
        except:
            # exception for lists
            return len(object_list)
        return estimated_count

class MyAdmin(admin.ModelAdmin):
    def get_queryset(self, request):
        qs = super(MyAdmin, self).get_queryset(request)
        qs.count = CountProxy()
        return qs

But I don#t know how to access the original queryset within my CountProxy class. Any idea? I know I can overwrite the whole changelist view through get_changelist. But that involves a lot of duplicating code from Django's repo.

Upvotes: 8

Views: 3146

Answers (4)

Constantinos Petrakis
Constantinos Petrakis

Reputation: 91

If you're trying to make the admin page faster for large counts you can define a wrapper of models.Admin that as queryset returns a queryset that instead of the normal count uses an approximate one only when results ain't filetered. This would work for Postgres 12 and Django 4.1.7:

class FastCountAdmin(admin.ModelAdmin):
    class FastCountQuerySet(QuerySet):
        def count(self):
            """
            Override count queries (performed by Django ORM) to display approximate value.
            This will speed up count in the admin interface.
            """

            if self._result_cache is not None:
                return len(self._result_cache)

            query = self.query
            if not (query.group_by or query.where or query.distinct):
                cursor = connection.cursor()
                cursor.execute("SELECT reltuples FROM pg_class WHERE relname = %s", 
                    [self.model._meta.db_table])
                return int(cursor.fetchone()[0])
            else:
                return self.query.get_count(using=self.db)

    def get_queryset(self, request):
        qs = super().get_queryset(request)
        return FastCountAdmin.FastCountQuerySet(qs.model, using=qs.db)

Then you can use it like that:

class MyHugeModelAdmin(FastCountAdmin):
    model = MyHugeModel

Upvotes: 0

radtek
radtek

Reputation: 36350

This is what worked for me with postgres and Django 2.2.x

from django.db.models.query import QuerySet
from django.db import connection

class FastCountQuerySet(QuerySet):
    """
    Fast count (estimate) queryset to speedup count
    """
    def count(self):
        """
        Override count queries (performed by Django ORM) to display approximate value.
        This will speed up count i.e. in the admin interface.
        """
        if self._result_cache is not None:
            return len(self._result_cache)

        query = self.query
        if not (query.group_by or query.where or query.distinct):
            # cursor = connections[self.db].cursor()
            cursor = connection.cursor()
            cursor.execute("SELECT reltuples FROM pg_class WHERE relname = %s", [self.query.model._meta.db_table])
            n = int(cursor.fetchone()[0])
            if n >= 1000:
                return n  # exact count for small tables
            else:
                return self.query.get_count(using=self.db)
        else:
            return self.query.get_count(using=self.db)


class CustomManager(models.Manager):
    """
    Custom db manager
    """
    def get_queryset(self):
        return FastCountQuerySet(self.model)

And finally override your model manager:

class YourModel(models.Model):
    objects = CustomManager()

Upvotes: 5

Seif
Seif

Reputation: 1097

I did something similar before so I can help.

I defined a custom queryset class:

class MyQuerySet(QuerySet):

    def count(self):
        """
        Override count queries (performed by Django ORM) to display approximate value.
        This will speed the admin interface.

        """
        if self._result_cache is not None and not self._iter:
            return len(self._result_cache)

        query = self.query
        if not (query.group_by or query.having or query.distinct):
            cursor = connections[self.db].cursor()
            cursor.execute("SHOW TABLE STATUS LIKE '%s';" % self.model._meta.db_table)
            return cursor.fetchall()[0][4]
        else:
            return self.query.get_count(using=self.db)

Then defined a custom model manager:

class MyManager(models.Manager):

    def get_query_set(self):
        return MyQuerySet(self.model)

Then used it in my model:

class MyModel(models.Model):
    objects = MyManager()

Upvotes: 2

Shang Wang
Shang Wang

Reputation: 25549

I could be wrong, but could you pass qs as an instance attribute for CountProxy?

class CountProxy:
    def __init__(self, query):
        self.query = query

    def __call__(self):
        # you've already had the query here, do something with self.query

class MyAdmin(admin.ModelAdmin):
    def get_queryset(self, request):
        qs = super(MyAdmin, self).get_queryset(request)
        qs.count = CountProxy(qs)
        return qs

Upvotes: 4

Related Questions