jmur
jmur

Reputation: 149

Why N+1 problem is enormous in DRF and how can I solve it?

I have site that was powered on vanilla django framework. I created a custom manager for my models to reduce N+1 problem and it worked very well. But when I moved to DRF, I just can't handle it with usual select_related and prefetch_related methods. Number of queries has increased dramatically. For example, here is my UserPostListView that should display 10 user's posts:

class UserPostListView(generics.ListAPIView):
    serializer_class = PostSerializer
    comment_serializer_class = CommentSerializer

    def get_queryset(self):
        slug = self.kwargs['slug']
        return Post.objects.detail().filter(author__slug=slug).order_by('-post_date')

    def list(self, request, *args, **kwargs):
        response = super().list(request, *args, **kwargs)
        data = response.data
        posts = data['results']
        for post in posts:
            comments = Comment.objects.detail().filter(post_id=post['id'], comment_to_reply=None)[:5]
            serialized_comments = self.comment_serializer_class(comments, many=True).data
            post['comments'] = serialized_comments
        data['results'] = posts
        response.data = data
        return response

It generates around 130 queries. I tried move this comment functionality to the serializer. It didn't help at all.

Ideally I see it working smth like that:

class PostSerializer(TaggitSerializer, serializers.ModelSerializer):
    author = UserSerializer(read_only=True)
    tags = CustomTagListSerializerField(required=False, allow_empty=True, allow_null=True)
    connected_subs = UserFilteredSubscriptionsField(many=True, required=False)
    post_media = PostMediaSerializer(many=True, required=False)

    likes_count = serializers.SerializerMethodField()
    views_count = serializers.SerializerMethodField()
    comments_count = serializers.SerializerMethodField()
    calculate_grid = serializers.SerializerMethodField()

    def get_comments(self, instance):
        comments = Comment.objects.detail().filter(post=instance, comment_to_reply=None)[:5]
        comments_serializer = CommentSerializer(comments, many=True)
        return comments_serializer.data

    def to_representation(self, instance):
        representation = super().to_representation(instance)
        representation['comments'] = self.get_comments(instance)
        return representation

But actually it generates even more queries.

For some reasone, if I override it in views, it works well and generates only 30:

    def get(self, request, *args, **kwargs):
        post_instance = self.get_object()
        comments = Comment.objects.detail().filter(post=post_instance, comment_to_reply=None)[:5]
        post_serializer = self.serializer_class(post_instance)
        comments_serializer = self.comment_serializer_class(comments, many=True)
        return Response({
            'post': post_serializer.data,
            'comments': comments_serializer.data
        })

Upvotes: 1

Views: 81

Answers (0)

Related Questions