Django admin change form load quite slow

Question

One of my Django websites has following database models: In Django App “common”:

class Collection(models.Model):
    name = models.CharField(max_length = 255, unique = True)
    _short_name = models.CharField(db_column="short_name", max_length = 32, blank=True)

class Particle(models.Model):
    content = models.TextField(blank=False)
    owner = models.ForeignKey(Collection)
    order = models.IntegerField(null=True, blank=True)

In Django App “sitcom”:

class Media(models.Model):
    name = models.CharField(max_length = 248)
    _short_name = models.CharField(db_column="short_name", max_length = 32, blank=True)
    capital = models.CharField(max_length = 1)
    description = models.TextField(blank=True)
    progress = models.CharField(max_length = 32, blank=True, null=True)

class Relation(models.Model):
    name = models.CharField(max_length = 128)
    _short_name = models.CharField(db_column="short_name", max_length = 32, blank=True)
    description = models.TextField(blank=True)
    parent = models.ForeignKey('self', blank=True, null=True)
    order = models.IntegerField(blank=True, null=True)
    particle = models.ForeignKey(Particle, blank=True, null=True)
    media = models.ForeignKey(Media, blank=True, null=True)

In short, model class Relation has 3 foreign keys to other tables. The problem is, when I use Django Admin to change a single Relation, the page (change_form) loads rather slowly. Later, I changed model class Relation as following:

class Relation(models.Model):
    name = models.CharField(max_length = 128)
    _short_name = models.CharField(db_column="short_name", max_length = 32, blank=True)
    description = models.TextField(blank=True)
    order = models.IntegerField(blank=True, null=True)
    parent_id = models.IntegerField(blank=True, null=True)
    particle_id = models.IntegerField(blank=True, null=True)
    media_id = models.IntegerField(blank=True, null=True)

The modification changed Foreign Keys to IntegerFields, so it disabled some of the magics inside Django ORM system, and now the change form page loads really fast. My question is, what is the “disabled magics inside django orm”? what has the potential to cause the problem?

Answer 1

Django now has support for autocomplete_fields which uses Select2 for input. You can configure as follows:

from django.contrib import admin

class RelationAdmin(admin.ModelAdmin):
       autocomplete_fields = ('Media','Particle',)

admin.site.register(Relation, RelationAdmin)

Answer 2

Another possibility:

As mentioned by @yuji-tomita-tomita elsewhere on this page, the admin change_form renders a <select> element for a ForeignKey field.

This is done using the Select widget.

By default, each <option> in the <select> element has a text label which is based on the __str__() method for the model that the ForeignKey points to. This can be seen in the source for forms.models.ModelChoiceField.

So, if your Model.__str__() method uses related fields, you'll get additional database queries when the admin form populates the select options.

For example, based on the OP, suppose we would have this:

class Particle(models.Model):
    ...
    owner = models.ForeignKey(Collection, on_delete=models.CASCADE)
    ...

    def __str__(self):
        return 'particle owned by {}'.format(self.owner.name)

Then the admin would need to query the database to get the related Collection.name value for each Particle option.

The admin does not do this efficiently.

If you have many Particle objects, that can become very expensive very quickly.

Answer 3

In my case the slowness was caused primarily by the outdated django-debug-toolbar (v1.7). debug_toolbar.middleware.DebugToolbarMiddleware would cause the admin page to literally take 20 minutes to load if it contained a ForeignKey field with a couple hundred choices. Upgrading the toolbar to v1.8 solved everything. Maybe this helps someone.

Answer 4

If a ForeignKey Model has too many records it might freeze your edit form. The best way to start, is to limit the fields that should be displayed on the form and gradually/one-by-one add other fields, checking which field makes the form slow.

from django.contrib import admin
class RelationAdmin(admin.ModelAdmin):
       fields = ('name',)
admin.site.register(Relation, RelationAdmin)

Then after adding a field that causes the problem, e.g. Media, the form would freeze again. Therefore, if you still need this field on the form you can use Vishal Shah's answer with raw_id_fields = ('Media', )

Answer 5

In admin.py

from django.contrib import admin

class RelationAdmin(admin.ModelAdmin):
       raw_id_fields = ('Media','Particle',)

admin.site.register(Relation, RelationAdmin)

This brings up a nice little UI element in the form and considerably improves performance since it doesn't have to load a huge number of options in the select box.

Answer 6

It is not the magic of django Orm. It is magic of Form. When you create a Foreign key in Model, then in ModelForm, a ModelChoiceField creates which has all choices of ForeignKey Model. And django Admin use all the properties of Form to create HTML. So use this code.

from django import forms
class RelationForm(forms.ModelForm):
    parent = forms.ChoiceField(required=False,
                              choices=Relation.objects.values_list('id', 'name'))
    particle = forms.ChoiceField(required=False,
                              choices=Particle.objects.values_list('id', 'content'))
    media = forms.ChoiceField(required=False,
                              choices=Media.objects.values_list('id', 'name'))

    class Meta:
        model = Relation 

In Admis Site

from django.contrib import admin
class RelationAdmin(admin.ModelAdmin):
    form = RelationForm
    model = Relation

You can also cache the choices pass in a form.

Answer 7

I'm willing to bet the issue is due to your ForeignKey. By default, django renders a <select> element for every foreign key.

If you have thousands of rows, this easily starts to bloat your HTML / DOM and I've noticed browsers starting to crap out at 20k items rendered in a <select> tag.

To fix it, look into overriding your admin form and not using the default widgets.