CODEWITHSUNDEEP
pythondjangoperformanceormdjango-rest-framework
MarkD
MarkD

Reputation: 4954

Django Rest Framework- retrieving a related field on reverse foreign key efficiently

I have the following models that represent a working group of users. Each working group has a leader and members:

class WorkingGroup(models.Model):
    group_name = models.CharField(max_length=255)
    leader = models.ForeignKey(User, null=True, on_delete=models.SET_NULL)

class WorkingGroupMember(models.Model):
    group = models.ForeignKey(WorkingGroup, on_delete=models.CASCADE)
    user = models.ForeignKey(User, on_delete=models.CASCADE)

In DRF, I want to efficiently retrieve all groups (there are several hundred) as an array of the following json objects:

{
    'id': <the_group_id>
    'group_name': <the_group_name>
    'leader': <id_of_leader>
    'members': [<id_of_member_1>, <id_of_member_2>, ...]
}

To do so, I have set up the following serializer:

class WorkingGroupSerializer(serializers.ModelSerializer):
    members = serializers.SerializerMethodField()
    class Meta:
        model = WorkingGroup
        fields = ('id', 'group_name', 'leader', 'members',)

    def get_members(self, obj):
        return obj.workinggroupmember_set.all().values_list('user_id', flat=True)

So that in my view, I can do something like:

groups = WorkingGroup.objects.all().prefetch_related('workinggroupmember_set')
group_serializer = WorkingGroupSerializer(groups, many=True)

This works, and gives the desired result, however I am finding it does not scale well at all, as the prefetching workinggroupmember_set does not seem to be used inside of the get_members method (Silky is showing a single query to grab all WorkingGroup objects, and then a query for each workinggroupmember_set call in the get_members method). Is there a way to set up the members field in the serializer to grab a flattened/single field version of workinggroupmember_set without using a SerializerMethodField? Or some other way of doing this that lets me properly use prefetch?

Upvotes: 5

Views: 2391

Answers (2)

Alberto Chiusole
Alberto Chiusole

Reputation: 2804

In a recent project with DRF v3.9.1 and django 2.1, I needed to recursively expose all the children of an object, by having only a direct connection to the parent, which could have had multiple children.

Before, if I was to request the "tree" of an object, I was getting:

{
    "uuid": "b85385c0e0a84785b6ca87ce50132659",
    "name": "a",
    "parent": null
}

By applying the serialization shown below I get:

{
    "uuid": "b85385c0e0a84785b6ca87ce50132659",
    "name": "a",
    "parent": null
    "children": [
        {
            "uuid": "efd26a820b4e4f7c8e56c812a7791fcb",
            "name": "aa",
            "parent": "b85385c0e0a84785b6ca87ce50132659"
            "children": [
                {
                    "uuid": "ca2441fc7abf49b6aa1f3ebbc2dae251",
                    "name": "aaa",
                    "parent": "efd26a820b4e4f7c8e56c812a7791fcb"
                    "children": [],
                }
            ],
        },
        {
            "uuid": "40e09c85775d4f1a8578bba9c812df0e",
            "name": "ab",
            "parent": "b85385c0e0a84785b6ca87ce50132659"
            "children": [],
        }
    ],
}

Here is the models.py of the recursive object:

class CategoryDefinition(BaseModelClass):
    name = models.CharField(max_length=100)
    parent = models.ForeignKey('self', related_name='children',
                               on_delete=models.CASCADE,
                               null=True, blank=True)

To get all the reverse objects in the foreign key, apply a field to the serializer class:

class DeepCategorySerializer(serializers.ModelSerializer):
    children = serializers.SerializerMethodField()

    class Meta:
        model = models.CategoryDefinition
        fields = '__all__'

    def get_children(self, obj):
        return [DeepCategorySerializer().to_representation(cat) for cat in obj.children.all()]

Then apply this serializer to a DRF view function or generics class, such as:

re_path(r'categories/(?P<pk>[\w\d]{32})/',
        generics.RetrieveUpdateDestroyAPIView.as_view(
            queryset=models.CategoryDefinition.objects.all(),
            serializer_class=serializers.DeepCategorySerializer),
        name='category-update'),

Upvotes: 0

Sardorbek Imomaliev
Sardorbek Imomaliev

Reputation: 15400

Problem here that you are doing values_list on top of all which nullifies your prefetch_related. There is currently no way to do prefetch with values_list see https://code.djangoproject.com/ticket/26565. What you can do is to transition this into python code instead of SQL

class WorkingGroupSerializer(serializers.ModelSerializer):
    members = serializers.SerializerMethodField()
    class Meta:
        model = WorkingGroup
        fields = ('id', 'group_name', 'leader', 'members',)

    def get_members(self, obj):
        return [wgm.user_id for wgm in obj.workinggroupmember_set.all()]

Upvotes: 5

Related Questions